Image encoding and decoding method and apparatus generating an angular intra prediction mode

ABSTRACT

Provided is an image decoding method including: generating an intra prediction value of a current sample based on a position of the current sample in a current block and an intra prediction mode of the current block; determining a sample value of at least one filtering reference sample to be filtered and a first weight with respect to the filtering reference sample and a second weight with respect to the intra prediction value of the current sample; generating a filtered prediction sample value of the current sample based on the determined sample value of the filtering reference sample to be filtered, the intra prediction value of the current sample, the first weight with respect to the filtering reference sample, and the second weight with respect to the intra prediction value of the current sample; and generating a prediction block of the current block including the filtered prediction sample value.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. application Ser. No.18/151,570, filed on Jan. 9, 2023, which is a continuation of U.S.application Ser. No. 16/651,809, filed on Mar. 27, 2020, which is nowU.S. Pat. No. 11,695,930, issued on Jul. 4, 2023, which is a NationalStage Entry of International Application No. PCT/KR2018/011390, filed onSep. 27, 2018, which claims priority to U.S. Provisional PatentApplication No. 62/564,981, filed on Sep. 28, 2017, in the U.S. Patentand Trademark Office, the disclosures of which are incorporated byreference herein in their entireties.

TECHNICAL FIELD

A method and apparatus according to an embodiment may encode or decodean image by using a coding unit in various shapes included in the image.The method and apparatus according to an embodiment include a method andapparatus for intra prediction.

BACKGROUND ART

As hardware for reproducing and storing high resolution or high qualityimage content is being developed and supplied, the need for a codec foreffectively encoding or decoding the high resolution or high qualityimage content is increasing. Encoded image content may be reproduced bybeing decoded. Recently, methods of effectively compressing highresolution or high quality image content have been implemented. Forexample, an efficient image compressing method through a process ofarbitrarily processing an image to be encoded has been implemented.

Various data units may be used to compress an image, and an inclusionrelationship may exist between these data units. To determine a size ofa data unit used for image compressing, the data unit may be splitaccording to various methods. After an optimized data unit is determinedaccording to the characteristics of the image, encoding and decoding ofthe image may be performed.

DESCRIPTION OF EMBODIMENTS Solution to Problem

According to an embodiment, an image decoding method includes:obtaining, from a bitstream, information about a transform coefficientof a current block; generating an intra prediction value of a currentsample based on a position of the current sample in the current blockand an intra prediction mode of the current block; determining, based onthe position of the current sample in the current block, a sample valueof at least one filtering reference sample to be filtered and a firstweight with respect to the filtering reference sample and a secondweight with respect to the intra prediction value of the current sample,and generating a filtered prediction sample value of the current samplebased on the determined sample value of the filtering reference sampleto be filtered, the intra prediction value of the current sample, thefirst weight with respect to the filtering reference sample, and thesecond weight with respect to the intra prediction value of the currentsample; generating a prediction block of the current block, theprediction block including the filtered prediction sample value of thecurrent sample; obtaining a residual block of the current block based onthe obtained information about the transform coefficient of the currentblock; and reconstructing the current block based on the predictionblock of the current block and the residual block of the current block.

The generating of the intra prediction value of the current sample basedon the position of the current sample in the current block and the intraprediction mode of the current block may include: determining anoriginal reference sample corresponding to the current sample based onthe position of the current sample and the intra prediction mode of thecurrent block; and generating the intra prediction value of the currentsample based on a sample value of the original reference sample.

The first weight with respect to the filtering reference sample may bedetermined based on a distance between the filtering reference sampleand the current sample.

The greater the distance between the filtering reference sample and thecurrent sample, the smaller may be the first weight with respect to thefiltering reference sample.

The filtering reference sample may include at least one of an originalreference sample located in a horizontal direction of the current sampleand an original reference sample located in a vertical direction of thecurrent sample.

When the intra prediction mode of the current block is an angular mode,

-   -   the filtering reference sample may include at least one of        adjacent samples on the left and on the top of the current        sample located on a line passing by the current sample, and the        line may be in a prediction direction indicated by the angular        mode and an opposite direction to the prediction direction.

The determining, based on the position of the current sample in thecurrent block, of a sample value of at least one filtering referencesample to be filtered and a first weight with respect to the filteringreference sample and a second weight with respect to an intra predictionvalue of the current sample, and generating a filtered prediction samplevalue of the current sample based on the determined sample value of the(filtering reference sample to be filtered, the intra prediction valueof the current sample, the first weight with respect to the filteringreference sample, and the second weight with respect to the intraprediction value of the current sample may include: determining at leastone second intra prediction mode; and by using the determined at leastone second intra prediction mode, determining, based on the position ofthe current sample in the current block, a sample value of at least onefiltering reference sample to be filtered and a first weight withrespect to the filtering reference sample and a second weight withrespect to the intra prediction value of the current sample, andgenerating a filtered prediction sample value of the current samplebased on the determined sample value of the (filtering reference sampleto be filtered, the intra prediction value of the current sample, thefirst weight with respect to the filtering reference sample, and thesecond weight with respect to the intra prediction value of the currentsample.

The at least one second intra prediction mode may be determined in unitsof pictures or in units of blocks.

The at least one second intra prediction mode may be determined to be atleast one of the intra prediction mode, an intra prediction modeindicating an opposite direction to a prediction direction indicated bythe intra prediction mode, a horizontal mode, and a vertical mode.

The first weight and the second weight may be normalized values.

The determining, based on the position of the current sample in thecurrent block, a sample value of at least one filtering reference sampleto be filtered and a first weight with respect to the filteringreference sample and a second weight with respect to the intraprediction value of the current sample, and generating a filteredprediction sample value of the current sample based on the determinedsample value of the filtering reference sample to be filtered, the intraprediction value of the current sample, the first weight with respect tothe filtering reference sample, and the second weight with respect tothe intra prediction value of the current sample may include: when theintra prediction mode is a predetermined intra prediction mode,determining, based on the position of the current sample in the currentblock, a sample value of at least one filtering reference sample to befiltered and a first weight with respect to the filtering referencesample and a second weight with respect to the intra prediction value ofthe current sample, and generating a filtered prediction sample value ofthe current sample based on the determined sample value of the filteringreference sample to be filtered, the intra prediction value of thecurrent sample, the first weight with respect to the filtering referencesample, and the second weight with respect to the intra prediction valueof the current sample.

According to an embodiment, an image encoding method includes:generating an intra prediction value of a current sample based on aposition of the current sample in a current block and an intraprediction mode of the current block; determining, based on the positionof the current sample in the current block, a sample value of at leastone filtering reference sample to be filtered and a first weight withrespect to the filtering reference sample and a second weight withrespect to the intra prediction value of the current sample, andgenerating a filtered prediction sample value of the current samplebased on the determined sample value of the filtering reference sampleto be filtered, the intra prediction value of the current sample, thefirst weight with respect to the filtering reference sample, and thesecond weight with respect to the intra prediction value of the currentsample; generating a prediction block of the current block, theprediction block including the filtered prediction sample value of thecurrent sample; and encoding information about a transform coefficientof the current block based on the prediction block of the current block.

According to an embodiment, an image decoding apparatus includes aprocessor configured to: obtain, from a bitstream, information about atransform coefficient of a current block; generate an intra predictionvalue of the current sample based on the position of the current samplein the current block and an intra prediction mode of the current block;determine, based on the position of the current sample in the currentblock, a sample value of at least one filtering reference sample to befiltered and a first weight with respect to the filtering referencesample and a second weight with respect to the intra prediction value ofthe current sample, and generate a filtered prediction sample value ofthe current sample based on the determined sample value of the filteringreference sample to be filtered, the intra prediction value of thecurrent sample, the first weight with respect to the filtering referencesample, and the second weight with respect to the intra prediction valueof the current sample; generate a prediction block of the current block,the prediction block including the filtered prediction sample value ofthe current sample; obtain a residual block of the current block basedon the obtained information about the transform coefficient of thecurrent block; and reconstruct the current block based on the predictionblock of the current block and the residual block of the current block.

A computer program for the image decoding method according to anembodiment of the present disclosure may be recorded on acomputer-readable recording medium.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1A is a block diagram of an image decoding apparatus according tovarious embodiments.

FIG. 1B is a flowchart of an image decoding method according to variousembodiments.

FIG. 1C is a flowchart of an image decoding method according to variousembodiments.

FIG. 1D is a block diagram of an image decoder according to variousembodiments.

FIG. 2A is a block diagram of an image encoding apparatus according tovarious embodiments.

FIG. 2B is a flowchart of an image encoding method according to variousembodiments.

FIG. 2C is a flowchart of an image encoding method according to variousembodiments.

FIG. 2D is a block diagram of an image encoder according to variousembodiments.

FIG. 3 illustrates a process, performed by an image decoding apparatus,of determining at least one coding unit by splitting a current codingunit, according to an embodiment.

FIG. 4 illustrates a process, performed by an image decoding apparatus,of determining at least one coding unit by splitting a non-squarecurrent coding unit, according to an embodiment.

FIG. 5 illustrates a process, performed by an image decoding apparatus,of splitting a coding unit based on at least one of block shapeinformation and split shape mode information, according to anembodiment.

FIG. 6 illustrates a method, performed by an image decoding apparatus,of determining a predetermined coding unit from among an odd number ofcoding units, according to an embodiment.

FIG. 7 illustrates an order of processing, performed by an imagedecoding apparatus, a plurality of coding units when the plurality ofcoding units are determined by splitting a current coding unit,according to an embodiment.

FIG. 8 illustrates a process, performed by an image decoding apparatus,of determining that a current coding unit is to be split into an oddnumber of coding units, when the coding units are not processable in apredetermined order, according to an embodiment.

FIG. 9 illustrates a process, performed by an image decoding apparatus,of determining at least one coding unit by splitting a first codingunit, according to an embodiment.

FIG. 10 illustrates that a shape into which a second coding unit issplittable is restricted when the second coding unit having a non-squareshape, which is determined by an image decoding apparatus by splitting afirst coding unit, satisfies a predetermined condition, according to anembodiment.

FIG. 11 illustrates a process, performed by an image decoding apparatus,of splitting a square coding unit when split shape mode informationindicates that the square coding unit is not to be split into foursquare coding units, according to an embodiment.

FIG. 12 illustrates that a processing order between a plurality ofcoding units may be changed depending on a process of splitting a codingunit, according to an embodiment.

FIG. 13 illustrates a process of determining a depth of a coding unit asa shape and size of the coding unit change, when the coding unit isrecursively split such that a plurality of coding units are determined,according to an embodiment.

FIG. 14 illustrates depths that are determinable based on shapes andsizes of coding units, and part indexes (PIDs) that are fordistinguishing the coding units, according to an embodiment.

FIG. 15 illustrates that a plurality of coding units are determinedbased on a plurality of predetermined data units included in a picture,according to an embodiment.

FIG. 16 illustrates a processing block serving as a unit for determininga determination order of reference coding units included in a picture,according to an embodiment.

FIG. 17 is a diagram for describing intra prediction modes according toan embodiment.

FIG. 18 is a diagram for describing a method, performed by an imagedecoding apparatus, of generating a sample reconstructed using originalreference samples, according to an embodiment of the present disclosure.

FIGS. 19A and 19B are diagrams for describing a method, performed by animage decoding apparatus, of generating a sample reconstructed usingoriginal reference samples in a prediction direction of an intraprediction mode of a current block, according to an embodiment of thepresent disclosure.

FIG. 20 is a diagram for describing a method, performed by an imagedecoding apparatus, of generating a sample reconstructed using originalreference samples, according to an embodiment of the present disclosure.

FIG. 21 is a diagram for describing a process, performed by an imagedecoding apparatus, of performing intra prediction on a current block byusing original reference samples and a reconstructed sample, accordingto an embodiment of the present disclosure.

FIG. 22 is a diagram for describing a process, performed by an imagedecoding apparatus, of performing weighted prediction by using anoriginal reference sample and reconstructed reference samples of a leftadjacent line and an upper adjacent line.

FIG. 23 is a diagram for describing a process, performed by an imagedecoding apparatus, of performing weighted prediction by using aprediction value generated by performing intra prediction using anoriginal reference sample and reconstructed reference samples of a leftadjacent line and an upper adjacent line.

FIG. 24 is a diagram for describing a process, performed by an imagedecoding apparatus, of performing position-based intra prediction on acurrent sample when an intra prediction mode of a current block is oneof a DC mode, a planar mode, and a vertical mode.

FIG. 25 is a diagram for describing a process, performed by an imagedecoding apparatus, of performing position-based intra prediction on acurrent sample when an intra prediction mode of a current block is adiagonal mode in a bottom left direction.

FIG. 26 is a diagram for describing a process, performed by an imagedecoding apparatus, of performing position-based intra prediction on acurrent sample when an intra prediction mode of a current block is adiagonal mode in a top right direction.

FIG. 27 is a diagram for describing a process, performed by an imagedecoding apparatus, of performing position-based intra prediction on acurrent sample when an intra prediction mode of a current block is anangular mode adjacent to a diagonal mode in a bottom left direction.

FIG. 28 is a diagram for describing a process, performed by an imagedecoding apparatus, of performing position-based intra prediction on acurrent sample when an intra prediction mode of a current block is anangular mode adjacent to a diagonal mode in a top right direction.

FIG. 29 is a diagram for describing that an encoding (decoding) orderamong coding units is determined to be in a forward or backwarddirection based on an encoding order flag and that a right or upperreference line may be used for intra prediction according to thedetermined encoding (decoding), according to an embodiment of thepresent disclosure.

BEST MODE

According to various embodiments, an image decoding method includes:obtaining, from a bitstream, information about a transform coefficientof a current block; generating an intra prediction value of a currentsample based on a position of the current sample in the current blockand an intra prediction mode of the current block; determining, based onthe position of the current sample in the current block, a sample valueof at least one filtering reference sample to be filtered and a firstweight with respect to the filtering reference sample and a secondweight with respect to the intra prediction value of the current sample,and generating a filtered prediction sample value of the current samplebased on the determined sample value of the filtering reference sampleto be filtered, the intra prediction value of the current sample, thefirst weight with respect to the filtering reference sample, and thesecond weight with respect to the intra prediction value of the currentsample; generating a prediction block of the current block, theprediction block including the filtered prediction sample value of thecurrent sample; obtaining a residual block of the current block based onthe obtained information about the transform coefficient of the currentblock; and reconstructing the current block based on the predictionblock of the current block and the residual block of the current block.

According to various embodiments, an image encoding method includes:generating an intra prediction value of a current sample based on aposition of the current sample in a current block and an intraprediction mode of the current block; determining, based on the positionof the current sample in the current block, a sample value of at leastone filtering reference sample to be filtered and a first weight withrespect to the filtering reference sample and a second weight withrespect to the intra prediction value of the current sample, andgenerating a filtered prediction sample value of the current samplebased on the determined sample value of the filtering reference sampleto be filtered, the intra prediction value of the current sample, thefirst weight with respect to the filtering reference sample, and thesecond weight with respect to the intra prediction value of the currentsample; generating a prediction block of the current block, theprediction block including the filtered prediction sample value of thecurrent sample; and encoding information about a transform coefficientof the current block based on the prediction block of the current block.

According to various embodiments, an image decoding apparatus includes aprocessor configured to: obtain, from a bitstream, information about atransform coefficient of a current block; generate an intra predictionvalue of the current sample based on the position of the current samplein the current block and an intra prediction mode of the current block;determine, based on the position of the current sample in the currentblock, a sample value of at least one filtering reference sample to befiltered and a first weight with respect to the filtering referencesample and a second weight with respect to the intra prediction value ofthe current sample, and generate a filtered prediction sample value ofthe current sample based on the determined sample value of the filteringreference sample to be filtered, the intra prediction value of thecurrent sample, the first weight with respect to the filtering referencesample, and the second weight with respect to the intra prediction valueof the current sample; generate a prediction block of the current block,the prediction block including the filtered prediction sample value ofthe current sample; obtain a residual block of the current block basedon the obtained information about the transform coefficient of thecurrent block; and reconstruct the current block based on the predictionblock of the current block and the residual block of the current block.

A program for implementing the method according to various embodimentsmay be included in a computer-readable recording medium.

MODE OF DISCLOSURE

The advantages and features of embodiments disclosed and methods ofachieving the advantages and features will be described more fully withreference to the accompanying drawings, in which the embodimentsprovided below shown. The present disclosure may, however, be embodiedin many different forms and should not be construed as being limited tothe embodiments set forth herein; rather these embodiments are providedso that this disclosure will be thorough and complete, and will fullyconvey the concept of the disclosure to one of ordinary skill in theart.

Hereinafter, the terms used in the specification will be brieflydescribed, and then the embodiments disclosed will be described indetail.

The terms used in this specification are those general terms currentlywidely used in the art in consideration of functions in regard to thedisclosure, but the terms may vary according to the intention of thoseof ordinary skill in the art, precedents, or new technology in the art.Also, specified terms may be selected by the applicant, and in thiscase, the detailed meaning thereof will be described in the detaileddescription of the disclosure. Thus, the terms used in the specificationshould be understood not as simple names but based on the meaning of theterms and the overall description of the disclosure.

An expression used in the singular encompasses the expression of theplural, unless it explicitly specifies the singular.

Throughout the specification, it will also be understood that when acomponent “includes” an element, unless there is another oppositedescription thereto, it should be understood that the component does notexclude another element but may further include another element.

In addition, terms such as a “ . . . unit” used in the specificationrefer to a software or hardware component, and a “ . . . unit” performscertain functions. However, a “ . . . unit” is not limited to softwareor hardware. A “ . . . unit” may be configured in an addressable storagemedium or to reproduce one or more processors. Thus, for example, a “ .. . unit” includes components such as software components,object-oriented software components, class components, task components,processes, functions, attributes, procedures, subroutines, segments ofprogram codes, drivers, firmware, microcodes, circuits, data, databases,data structures, tables, arrays, and variables. Functions provided inthe components and the “ . . . units” may be combined with a smallernumber of components and “ . . . units“or separated from additionalcomponents and” . . . units.”

According to an embodiment of the present disclosure, a “ . . . unit”may be implemented by a processor and a memory. The term “processor”should be interpreted broadly to include general-purpose processors,central processing units (CPUs), microprocessors, digital signalprocessors (DSPs), controllers, microcontrollers, state machines, andthe like. In some circumstances, a “processor” may refer to anapplication specific semiconductor (ASIC), a programmable logic device(PLD), a field programmable gate array (FPGA), or the like. The term“processor” may refer to a combination of processing devices such as,for example, a combination of a DSP and a microprocessor, a combinationof a plurality of microprocessors, a combination of one or moremicroprocessors in conjunction with a DSP core, or a combination of anyother such configuration.

The term “memory” should be interpreted broadly to include anyelectronic component capable of storing electronic information. The term“memory” may refer to various types of processor-readable media such asrandom access memory (RAM), read-only memory (ROM), non-volatile randomaccess memory (NVRAM), programmable read-only memory (PROM),erase-programmable read-only memory (EPROM), electrically erasable PROM(EEPROM), flash memory, magnetic or optical data storage, registers, andthe like. When a processor can read information from and/or writeinformation to a memory, the memory is said to be in electroniccommunication with the processor. A memory integrated in a processor isin electronic communication with the processor.

Hereinafter, an ‘image’ may refer to a still image such as a staticimage of a video or a moving image, that is, a motion image such as avideo itself.

Hereinafter, a ‘sample’ means data to be processed as data allocated toa sampling position of an image. For example, pixel values of a spatialdomain and transform coefficients on a transform region may be samples.A unit including at least one sample described above may be defined as ablock.

Hereinafter, embodiments will be described in detail with reference tothe accompanying drawings so that those of skill in the art may easilyimplement the embodiments. Also, elements not related to description areomitted in the drawings for clear description of the present disclosure.

Hereinafter, with reference to FIGS. 1 to 29 , an image encodingapparatus and an image decoding apparatus, and an image encoding methodand an image decoding method, according to an embodiment, will bedescribed. A method of determining a data unit of an image according toan embodiment will be described with reference to FIGS. 3 through 16 .An encoding or decoding method and apparatus for determining a filteringreference sample to be filtered and a weight of a filter, and adaptivelyperforming intra prediction based on the filtering reference sample andthe weight of the filter, according to an embodiment, will be describedwith reference to FIGS. 1, 2, and 17 through 29 .

Hereinafter, an encoding/decoding method and apparatus for adaptivelyperforming intra prediction based on various shapes of coding units,according to an embodiment of the present disclosure, will be describedwith reference to FIGS. 1 and 2 .

FIG. 1A is a block diagram of an image decoding apparatus according tovarious embodiments.

An image decoding apparatus 100 according to various embodiments mayinclude an obtainer 105, an intra predictor 110, and an image decoder115.

The obtainer 105, the intra predictor 110, and the image decoder 115 mayinclude at least one processor. In addition, the obtainer 105, the intrapredictor 110, and the image decoder 115 may include a memory storinginstructions to be executed by the at least one processor. The imagedecoder 115 may be implemented as a separate hardware from the obtainer105 and the intra predictor 110 or may include the obtainer 105 and theintra predictor 110.

The obtainer 105 may obtain, from a bitstream, information about atransform coefficient of a current block. The obtainer 105 may obtain,from a bitstream, information about a prediction mode of the currentblock and information about an intra prediction mode of the currentblock.

The information about a prediction mode of the current block obtained bythe obtainer 105 may include information indicating an intra mode or aninter prediction mode. Information about an intra prediction of thecurrent block may be information about an intra prediction mode appliedto the current block from among a plurality of intra prediction modes.For example, an intra prediction mode may be one of a DC mode, a planarmode, and at least one angular mode having a prediction direction. Anangular mode may include a horizontal mode, a vertical mode, and adiagonal mode, and a mode having a predetermined direction except for ahorizontal direction, a vertical direction, and a diagonal direction.For example, there may be 65 or 33 angular modes.

The intra predictor 110 may be activated when a prediction mode of thecurrent block is an intra prediction mode.

The intra predictor 110 may determine an original referencesample-related value based on a position of a current sample in thecurrent block and an intra prediction mode of the current block. Thatis, the intra predictor 110 may determine at least one originalreference sample from among reference samples, based on the position ofthe current sample in the current block and the intra prediction mode ofthe current block, and determine an original reference sample-relatedvalue based on the determined at least one original reference sample.Original reference sample refers to samples of a neighboring block of acurrent block, and the original reference sample may include a sample ofa block adjacent to a left side of the current block or a sample of ablock adjacent to an upper portion of the current block. For example, anoriginal reference sample may include a sample on a predetermined linein a vertical direction adjacent to the left of the current block or asample on a predetermined line in a horizontal direction adjacent to theupper portion of the current block. However, the original referencesample is not limited to include a sample of a block adjacent to theleft of the current block or a sample of a block adjacent to the upperportion of the current block, but may also include a sample of a blockadjacent to the upper portion of the current block or a sample of ablock adjacent to the right of the current block.

The intra predictor 110 may determine an intra prediction value withrespect to the current sample based on the position of the currentsample in the current block and the intra prediction mode of the currentblock. The intra predictor 110 may determine an original referencesample corresponding to the current sample based on the position of thecurrent sample and the intra prediction mode of the current block. Theintra predictor 110 may generate an intra prediction value of thecurrent sample based on a sample value of the original reference sample.The intra predictor 110 may determine at least one filtering referencesample to be filtered and a weight of a filter based on at least one ofan intra prediction mode of a current block, a position of a currentsample in the current block, and a position of a reference sample of thecurrent block, and may determine a filtering reference sample-relatedvalue based on the filtering reference sample and the weight of thefilter.

For example, when an intra prediction mode of a current block is one ofa planar mode, a DC mode, a horizontal mode, and a vertical mode, theintra predictor 110 may determine, as a filtering reference sample, atleast one of a sample adjacent to a top left corner of the currentblock, a sample that is adjacent to the current block and is in a topdirection of the current block, and a sample that is adjacent to thecurrent block and is located in a left direction of the current block.

When an intra prediction mode of a current block is an angular modeincluding an diagonal mode, the intra predictor 110 may determine, as afiltering reference sample, at least one of an adjacent sample on theleft and on the top of the current block, which is located on a linepassing by a current sample in the current block and a sample adjacentto a top left corner of the current block. Here, the line may be in aprediction direction indicated by an angular mode or an oppositedirection thereto.

The intra predictor 110 may determine the number of tabs of a filter tobe applied to a filtering reference sample based on at least one of anintra prediction mode of a current block and a size of the currentblock.

In addition, the intra predictor 110 may also determine a weight of afilter to be applied to a filtering reference sample based on a size ofa current block, without being limited to determining a weight of afilter to be applied to a filtering reference sample based on at leastone of an intra prediction mode of a current block, a position of acurrent sample in the current block, and a position of a referencesample.

The intra predictor 110 may determine some of samples within a referenceline adjacent to a current block, as a filtering reference sample, basedon a horizontal direction component and a vertical direction componentof a prediction direction specified by an intra prediction mode.

When an intra prediction mode of a current block is a predeterminedintra prediction mode, the intra predictor 110 may determine a filteringreference sample-related value based on a filtering reference sample tobe filtered and a weight of a filter, based on at least of the intraprediction mode of the current block, a position of a current sample inthe current block, and a position of a reference sample in the currentblock.

The intra predictor 110 may determine at least one intra predictionmode, and determine a filtering reference sample to be filtered and aweight of a filter based on at least one of a position of a currentsample in a current block and a position of a reference sample of thecurrent block, by using the determined at least one intra predictionmode. The intra predictor 110 may determine a filtering referencesample-related value based on the filtering reference sample and theweight of the filter.

Here, at least one intra prediction mode may be determined for eachpicture unit or in units of blocks. At least one intra prediction modemay be an intra prediction mode determined based on an intra predictionmode of a current block or may be a predetermined intra prediction mode.The predetermined intra prediction mode may be at least one of ahorizontal mode and a vertical mode.

The intra predictor 110 may obtain a prediction block of a currentblock, the prediction block including a prediction sample of the currentblock, based on at least one of an original reference sample-relatedvalue and a filtering reference sample-related value. For example, theintra predictor 110 may determine whether to perform intra prediction ona current sample by using both the original reference sample-relatedvalue and the filtering reference sample-related value, or may determineone of the original reference sample-related value and the filteringreference sample-related value, and determine whether to perform intraprediction on a current sample based on the determined value. The intrapredictor 110 may obtain a prediction block of the current block, theprediction block including a prediction sample of the current sample,based on the above determination.

The intra predictor 110 may determine a sample value of at least onefiltering reference sample to be filtered, a first weight with respectto the filtering reference sample, and a second weight with respect toan intra prediction value of a current sample, based on the position ofthe current sample in the current block. The first weight with respectto the filtering reference sample may be determined based on a distancebetween the filtering reference sample and the current sample. Forexample, the first weight may be determined based on a distance betweenthe filtering reference sample and the current sample, compared to asize of the current block. Here, the size of the current block may referto a height or width of the current block. The first weight may be thesmaller, the greater the distance between the filtering reference sampleand the current sample. The second weight may also be determined in asimilar manner to the first weight. The first weight and the secondweight may be normalized values.

Here, the filtering reference sample may include at least one of anoriginal reference sample located in a horizontal direction of thecurrent sample and an original reference sample located in a verticaldirection of the current sample. When an intra prediction mode of acurrent block is an angular mode, a filtering reference sample mayinclude at least one of adjacent samples on the left and on the top ofthe current block, which are located on a line passing by the currentsample. Here, the line may be in a prediction direction indicated by theangular mode or an opposite direction thereto.

The intra predictor 110 may generate a filtered prediction sample valueof a current sample based on a sample value of a filtering referencesample to be filtered, an intra prediction value of the current sample,a first weight with respect to the filtering reference sample, and asecond weight with respect to the intra prediction value of the currentsample. For example, only when an intra prediction mode of a currentblock is a predetermined intra prediction mode, the intra predictor 110may generate a filtered prediction sample value of a current samplebased on a sample value of a filtering reference sample to be filtered,an intra prediction value of the current sample, a first weight withrespect to the filtering reference sample, and a second weight withrespect to the intra prediction value of the current sample.

For example, the intra predictor 110 may determine at least one secondintra prediction mode, and may determine, by using the at least onesecond intra prediction mode, a sample value of at least one filteringreference sample to be filtered and a first weight with respect to thefiltering reference sample and a second weight with respect to an intraprediction value of the current sample based on a position of a currentsample in a current block, and generate a filtered prediction samplevalue of the current sample based on the determined sample value of thefiltering reference sample to be filtered, the intra prediction value ofthe current sample, the first weight with respect to the filteringreference sample, and the second weight with respect to the intraprediction value of the current sample. Here, at least one second intraprediction mode may be determined for each picture unit or in units ofblocks. At least one second intra prediction mode may be determined tobe at least one of an intra prediction mode of a current block, an intraprediction mode indicating an opposite direction to a predictiondirection indicated by the intra prediction mode of the current block, ahorizontal mode, and a vertical mode.

The intra predictor 110 may generate a prediction block of a currentblock, the prediction block including a filtered prediction sample valueof the current block.

The intra predictor 110 may obtain a prediction sample of a currentblock by performing filtering based on a first weight with respect to anoriginal reference sample-related value and a second weight with respectto a filtering reference sample-related value, and the originalreference sample-related value, and the filtering referencesample-related value.

The intra predictor 110 may determine a smaller second weight withrespect to a filtering reference sample-related value, the greater adistance from a filtering reference sample to a current sample.

The image decoder 115 may obtain a residual block of a current blockbased on information about a transform coefficient of the current block.That is, the image decoder 115 may obtain, from a bitstream, a residualsample about a residual block of a current block by performing inversequantization and inverse transformation based on information about atransform coefficient of the current block.

The image decoder 115 may reconstruct the current block based on aprediction block of the current block and the residual block of thecurrent block. The image decoder 115 may generate a reconstructionsample in the current block by using a sample value of a predictionsample in the prediction block of the current block and a sample valueof a residual sample in the residual block of the current block, andgenerate a reconstruction block of the current block based on thereconstruction sample.

Meanwhile, the image decoding apparatus 100 may obtain, from abitstream, flag information indicating whether intra prediction isadaptively performed based on a filtering reference sample and a weightof a filter, and may determine, based on the flag information, whetherto adaptively perform intra prediction based on the filtering referencesample and the weight of the filter. Here, the flag information may beobtained for each block, particularly, for each coding tree unit.

Also, the image decoding apparatus 100 may obtain flag information thatcommonly applies to a luminance component and chrominance component.Alternatively, the image decoding apparatus 100 may obtain flaginformation that applies to each of a luminance component or achrominance component.

Alternatively, the image decoding apparatus 100 may not obtain flaginformation from a bitstream but may determine whether to adaptivelyperform intra prediction based on a filtering reference sample and aweight of a filter. For example, when a prediction mode of a currentblock is a predetermined intra prediction mode, the image decodingapparatus 100 may determine to adaptively perform intra prediction basedon a filtering reference sample and a weight of a filter.

Alternatively, the image decoding apparatus 100 may not obtain flaginformation from a bitstream but may determine whether to adaptivelyperform intra prediction based on a filtering reference sample and aweight of a filter, by using information of a neighboring block. Forexample, the image decoding apparatus 100 may determine whether toadaptively perform intra prediction based on a filtering referencesample and a weight of a filter with respect to a current block based onflag information of a neighboring block of a current block, theneighboring block indicating whether intra prediction is adaptivelyperformed, based on the filtering reference sample and the weight of thefilter with respect to the neighboring block of the current block.

Alternatively, the image decoding apparatus 100 may determine whether toadaptively perform intra prediction, based on the filtering referencesample and the weight of the filter based on a size of the currentblock. For example, when a size of a current block is a predeterminedfirst block size, the image decoding apparatus 100 may adaptivelyperform intra prediction based on a filtering reference sample and aweight of a filter, and when a size of a current block is apredetermined second block size, the image decoding apparatus 100 maynot adaptively perform intra prediction based on a filtering referencesample and a weight of a filter but may perform intra predictionaccording to the related art.

The image decoding apparatus 100 may perform intra prediction bycombining encoding/decoding tools of intra prediction that are similarto encoding/decoding tools of adaptive intra prediction based on afiltering reference sample and a weight of a filter. Alternatively, theimage decoding apparatus 100 may assign a priority to a plurality ofencoding/decoding tools of intra prediction and perform intra predictionbased on the priorities among the encoding/decoding tools. That is, whenan encoding/decoding tool having a high priority is used, anencoding/decoding tool having a low priority may not be used; when anencoding/decoding tool having a high priority is not used, anencoding/decoding tool having a low priority may be used.

FIG. 1B is a flowchart of an image decoding method according to variousembodiments.

In operation S105, the image decoding apparatus 100 may obtaininformation about a transform coefficient of a current block.

In operation S110, the image decoding apparatus 100 may determine atleast one filtering reference sample to be filtered and a weight of afilter based on at least one of an intra prediction mode of the currentblock, a position of a current sample in the current block, and aposition of a reference sample of the current block, and may determine afiltering reference sample-related value based on the filteringreference sample to be filtered and the weight of the filter.

In operation S115, the image decoding apparatus 100 may determine anoriginal reference sample-related value based on the position of thecurrent sample in the current block and the intra prediction mode of thecurrent block.

In operation S120, the image decoding apparatus 100 may obtain aprediction block of the current block, the prediction block including aprediction sample of the current sample, based on at least one of theoriginal reference sample-related value and the filtering referencesample-related value.

In operation S125, the image decoding apparatus 100 may obtain aresidual block of the current block based on the information about thetransform coefficient of the current block.

In operation S130, the image decoding apparatus 100 may reconstruct thecurrent block based on the prediction block of the current block and theresidual block of the current block.

FIG. 1C is a flowchart of an image decoding method according to variousembodiments.

In operation S155, the image decoding apparatus 100 may obtaininformation about a transform coefficient of a current block.

In operation S160, the image decoding apparatus 100 may generate anintra prediction value with respect to a current sample based on aposition of the current sample in the current block and an intraprediction mode of the current block.

In operation S165, the image decoding apparatus 100 may determine asample value of at least one filtering reference sample to be filteredand a first weight with respect to the filtering reference sample and asecond weight with respect to an intra prediction value of the currentsample based on a position of the current sample in the current block,and generate a filtered prediction sample value of the current samplebased on the sample value of the filtering reference sample to befiltered, the intra prediction value of the current sample, the firstweight with respect to the filtering reference sample, and the secondweight with respect to the intra prediction value of the current sample.

In operation S170, the image decoding apparatus 100 may generate aprediction block of the current block, the prediction block includingthe filtered prediction sample value of the current sample.

In operation S175, the image decoding apparatus 100 may obtain aresidual block of the current block based on the information about thetransform coefficient of the current block.

In operation S180, the image decoding apparatus 100 may reconstruct thecurrent block based on the prediction block of the current block and theresidual block of the current block.

FIG. 1D is a block diagram of an image decoder 6000 according to variousembodiments.

The image decoder 6000 according to various embodiments performsoperations that are performed by the image decoder 115 of the imagedecoding apparatus 100 to encode image data.

Referring to FIG. 1D, an entropy decoder 6150 parses, from a bitstream6050, encoded image data to be decoded and encoding information neededfor decoding. Encoded image data is a quantized transform coefficient,and an inverse quantizer 6200 and an inverse transformer 6250reconstruct residue data from the quantized transform coefficient.

An intra predictor 6400 performs intra prediction on each block. Theintra predictor 6400 of FIG. 1D may correspond to the intra predictor110 of FIG. 1A.

An inter predictor 6350 performs inter prediction by using a referenceimage obtained from a reconstruction picture buffer 6300. As predictiondata and residue data with respect to each block generated in the intrapredictor 6400 or the inter predictor 6350 are added, data of a spatialdomain with respect to a block of a current image may be reconstructed,and a deblocker 6450 and a SAO performer 6500 may perform loop filteringon the reconstructed data of the spatial domain to output a filteredreconstruction image 6600. In addition, reconstruction images stored inthe reconstruction picture buffer 6300 may be output as a referenceimage.

In order for a decoder (not shown) of the image decoding apparatus 100to decode image data, the stepwise operations of the image decoder 6000according to various embodiments may be performed on each block.

FIG. 2A is a block diagram of an image encoding apparatus according tovarious embodiments.

An image encoding apparatus 150 according to various embodiments mayinclude an intra predictor 155 and an image encoder 160.

The intra predictor 155 and the image encoder 160 may include at leastone processor. In addition, intra predictor 155 and the image encoder160 may include a memory storing instructions to be executed by the atleast one processor. The image encoding apparatus 150 may be implementedas separate hardware from the intra predictor 155 and the image encoder160 or may include the intra predictor 155 and the image encoder 160.

The intra predictor 155 may determine a filtering reference sample to befiltered and a weight of a filter based on at least one of an intraprediction mode of a current block, a position of a current sample inthe current block, and a position of a reference sample of the currentblock, and may determine a filtering reference sample-related valuebased on the filtering reference sample to be filtered and the weight ofthe filter. The intra predictor 155 may determine an original referencesample-related value based on the position of the current sample in thecurrent block and the intra prediction mode of the current block.

The intra predictor 155 may generate a prediction block of the currentblock, the prediction block including a prediction sample of the currentsample, based on at least one of the original reference sample-relatedvalue and the filtering reference sample-related value.

Alternatively, the intra predictor 155 may determine an intra predictionvalue of the current sample based on the position of the current samplein the current block and the intra prediction mode of the current block.The intra predictor 155 may determine a sample value of at least onefiltering reference sample to be filtered, and a first weight withrespect to the filtering reference sample, and a second weight withrespect to the intra prediction value of the current sample, based onthe position of the current sample in the current block. The intrapredictor 155 may generate a filtered prediction sample value of thecurrent sample based on the sample value of the filtering referencesample to be filtered, the intra prediction value of the current sample,the first weight with respect to the filtering reference sample, and thesecond weight with respect to the intra prediction value of the currentsample. The intra predictor 155 may generate a prediction block of thecurrent block, the prediction block including the filtered predictionsample value of the current sample.

The image encoder 160 may encode information about a transformcoefficient of the current block based on the prediction block of thecurrent block. That is, the image encoder 160 may generate a residualblock of the current block based on an original block of the currentblock and the prediction block of the current block, and transform andquantize the residual block of the current block to encode theinformation about the transform coefficient of the current block. Theimage encoder 160 may encode information about a prediction mode of thecurrent block and information about the intra prediction mode of thecurrent block.

The image encoder 160 may generate a bitstream including the informationabout the transform coefficient of the current block and output thebitstream.

FIG. 2B is a flowchart of an image encoding method according to variousembodiments.

In operation S205, the image encoding apparatus 150 may determine atleast one filtering reference sample to be filtered and a weight of afilter based on at least one of an intra prediction mode of a currentblock, a position of a current sample in the current block, and aposition of a reference sample of the current block, and may determine afiltering reference sample-related value based on the filteringreference sample to be filtered and the weight of the filter.

In operation S210, the image encoding apparatus 150 may determine anoriginal reference sample-related value based on the position of thecurrent sample in the current block and the intra prediction mode of thecurrent block.

In operation S215, the image encoding apparatus 150 may generate aprediction block of the current block, the prediction block including aprediction sample of the current sample, based on at least one of theoriginal reference sample-related value and the filtering referencesample-related value.

In operation S220, the image encoding apparatus 150 may encodeinformation about a transform coefficient of the current block based onthe prediction block of the current block.

FIG. 2C is a flowchart of an image encoding method according to variousembodiments.

In operation S250, the image encoding apparatus 150 may generate anintra prediction value of a current sample based on the position of thecurrent sample in the current block and an intra prediction mode of thecurrent block.

In operation S255, the image encoding apparatus 150 may determine asample value of at least one filtering reference sample to be filtered,and a first weight with respect to the filtering reference sample and asecond weight with respect to the intra prediction value of the currentsample, based on the position of the current sample in the currentblock, and generate a filtered prediction sample value of the currentsample based on the determined sample value of the filtering referencesample to be filtered, the intra prediction value of the current sample,the first weight with respect to the filtering reference sample, and thesecond weight with respect to the intra prediction value of the currentsample.

In operation S260, the image encoding apparatus 150 may generate aprediction block of the current block, the prediction block includingthe filtered prediction sample value of the current sample.

In operation S265, the image encoding apparatus 150 may encodeinformation about a transform coefficient of the current block based onthe prediction block of the current block.

FIG. 2D is a block diagram of an image encoder according to variousembodiments.

An image encoder 7000 according to various embodiments performsoperations that are performed by the image encoder 160 of the imageencoding apparatus 150 to encode image data.

That is, an intra predictor 7200 performs intra prediction on each blockof a current image 7050, and an inter predictor 7150 performs interprediction on each block by using a reference image obtained from thecurrent image 7050 and a reconstruction picture buffer 7100.

Residue data may be generated by taking, from data about an encodedblock of the current image 7050, prediction data about each block outputfrom the intra predictor 7200 or the inter predictor 7150, and atransformer 7250 and a quantizer 7300 may perform transformation andquantization on the residue data to output a quantized transformcoefficient for each block. The intra predictor 7200 of FIG. 2D maycorrespond to the intra predictor 155 of FIG. 2A.

An inverse quantizer 7450 and an inverse transformer 7500 may performinverse quantization and inverse transformation on the quantizedtransform coefficient to reconstruct residue data of a spatial domain.The reconstructed residue data of the spatial domain may be added toprediction data about each block output from the intra predictor 7200 orthe intra prediction to be reconstructed to data of a spatial domainwith respect to a block of the current image 7050. A deblocker 7550 andan SAO performer perform in-loop filtering on the reconstructed data ofthe spatial domain to generate a filtered reconstruction image. Thegenerated reconstruction image is stored in the reconstruction picturebuffer 7100. Reconstruction images stored in the reconstruction picturebuffer 7100 may be used as a reference image for inter prediction ofother images. An entropy encoder 7350 may entropy-encode a quantizedtransform coefficient, and the entropy-encoded coefficient may be outputto a bitstream 7400.

In order for the image encoder 7000 according to various embodiments tobe applied to the image encoding apparatus 150, step-wise operations ofthe image encoder 7000 according to various embodiments may be performedon each block.

Hereinafter, splitting of a coding unit according to an embodiment ofthe present disclosure will be described in detail.

First, one picture may be split into one or more slices. One slice maybe a sequence of one or more coding tree units (CTUs). A conceptcontrasted with a CTU is a coding tree block (CTB).

A CTB refers to an N×N block including N×N samples (N is an integer).Each color component may be split into one or more CTBs.

When a picture has three sample arrays (sample arrays for Y, Cr, and Cbcomponents), a coding tree unit (CTU) is a unit including a coding treeblock of a luma sample and two coding tree blocks of chroma samplescorresponding thereto, and syntax structures used in encoding the lumasample and the chroma samples. When a picture is a monochrome image, acoding tree unit is a unit including a coding tree block of a monochromesample and syntax structures used to encode monochrome samples. When apicture is a picture encoded to a color plane separated into colorcomponents, a coding tree unit is a unit including syntax structuresused to encode the picture and samples of the picture.

One coding tree block (CTB) may be split into an M×N coding blocksincluding M×N samples (M and N are integers).

When a picture has sample arrays for Y, Cr, and Cb components, a codingunit (CU) is a unit including a coding block of a luma sample and twocoding blocks of chroma samples corresponding thereto, and syntaxstructures used in encoding the luma sample and the chroma samples. Whena picture is a monochrome image, a code unit is a unit including acoding block of a monochrome sample and syntax structures used to encodemonochrome samples. When a picture is a picture encoded to a color planeseparated into color components, a coding unit is a unit includingsyntax structures used to encode the picture and samples of the picture.

As described above, a coding tree block and a coding tree unit areconcepts that are distinguished from each other, and a coding block anda coding unit are concepts that are distinguished from each other. Thatis, a coding (tree) unit refers to a data structure including a coding(tree) block including a sample and a syntax structure correspondingthereto. However, one of ordinary skill in the art may understand that acoding (tree) unit or a coding (tree) block refers to a block of apredetermined size including a predetermined number of samples. Thus, acoding tree block and a coding tree unit or a coding block and a codingunit will be described below in the specification without distinguishingbetween them unless otherwise noted.

An image may be split into a coding tree unit (CTU). A size of a codingtree unit may be determined based on information obtained from abitstream. A shape of a coding tree unit may be a square of an equalsize. However, the disclosure is not limited thereto.

For example, information about a maximum size of a luma coding block maybe obtained from a bitstream. For example, a maximum size of a lumacoding block indicated by information about the maximum size of the lumacoding block may be one of 16×16, 32×32, 64×64, 128×128, and 256×256.

For example, information about a maximum size of a luma coding blockthat can be split into two and a luma block size difference may beobtained from a bitstream. The information about a luma block sizedifference may indicate a size difference between a luma coding treeunit and a luma coding tree block that can be split into two. Therefore,by combining the information about the maximum size of the luma codingblock that can be split into two and the information about the lumablock size difference, the information being obtained from a bitstream,a size of the luma coding tree unit may be determined. By using a sizeof a luma coding tree unit, a size of a chroma coding tree unit may alsobe determined. For example, when a Y:Cb:Cr ratio is 4:2:0 according to acolor format, a size of a chroma block may be half a size of a lumablock, and similarly, a size of a chroma coding tree unit may be half asize of a luma coding tree unit.

According to an embodiment, as information about a maximum size of aluma coding block that can be binary-split is obtained from a bitstream,the maximum size of the luma coding block that can be binary-split maybe variably determined. In contrast, a maximum size of a luma codingblock that can be ternary-split may be fixed. For example, a maximumsize of a luma coding block that can be ternary-split in an I slice maybe 32×32, and a maximum size of a luma coding block that can beternary-split in a P slice or a B slice may be 64×64.

In addition, a coding tree unit may be hierarchically split into codingunits based on split shape mode information obtained from a bitstream.As the split shape mode information, at least one of informationindicating whether quad split is performed, information indicatingwhether multi-split is performed, split direction information, and splittype information may be obtained from a bitstream.

For example, the information indicating whether quad split is performedmay indicate whether a current coding unit is to be quad-split(QUAD_SPLIT) or not quad-split.

When a current coding unit is not quad-split, the information indicatingwhether multi-split is performed may indicate whether the current codingunit is no longer split (NO_SPLIT) or binary-/ternary-split.

When the current coding unit is binary-split or ternary-split, the splitdirection information indicates that the current coding unit is split inone of a horizontal direction or a vertical direction.

When the current coding unit is split in a horizontal or verticaldirection, the split type information indicates that the current codingunit is binary-split or ternary-split.

According to the split direction information and the split typeinformation, a split mode of the current coding unit may be determined.A split mode corresponding to binary split of a current coding unit in ahorizontal direction may be determined to be a binary horizontal split(SPLIT_BT_HOR); a split mode corresponding to ternary split of a currentcoding unit in a horizontal direction may be determined to be a ternaryhorizontal split (SPLIT_TT_HOR); a split mode corresponding to binarysplit of a current coding unit in a vertical direction may be determinedto be a binary vertical split (SPLIT_BT_VER); and a split modecorresponding to ternary split of a current coding unit in a verticaldirection may be determined to be a ternary vertical split(SPLIT_TT_VER).

The image decoding apparatus 100 may obtain split shape mode informationfrom a bin string from a bitstream. A shape of a bitstream received bythe image decoding apparatus 100 may include a fixed length binary code,a unary code, a truncated unary code, a preset binary code or the like.A bin string is binary representation of information. A bin string mayconsist of at least one bit. The image decoding apparatus 100 may obtainsplit shape mode information corresponding to a bin string, based on asplitting rule. The image decoding apparatus 100 may determine whetherto quad-split a coding unit, whether not to split a coding unit, or asplitting direction and a splitting type, based on one bin string.

A coding unit may be equal to or smaller than a coding tree unit. Forexample, a coding tree unit is also a coding unit having a maximum size,and thus is a type of coding unit. When split shape mode informationabout a coding tree unit indicates that the coding tree unit is notsplit, a coding unit determined from the coding tree unit has an equalsize to that of the coding tree unit. When split shape mode informationabout a coding tree unit indicates that the coding tree unit is split,the coding tree unit may be split into coding units. Also, when splitshape mode information about a coding unit indicates that the codingtree unit is split, coding units may be split into smaller coding units.However, image splitting is not limited thereto, and a coding tree unitand a coding unit may not be distinguished. Splitting of a coding unitwill be described in more detail with reference to FIGS. 3 through 16 .

Also, one or more prediction blocks for prediction may be determinedfrom a coding unit. A prediction block may be equal to or smaller than acoding unit. Also, one or more transform blocks for transformation maybe determined from a coding unit. A transform block may be equal to orsmaller than a coding unit.

A shape and size of a transform block and of a prediction block may notbe related to each other.

According to another embodiment, prediction may be performed by using acoding unit as a prediction block. Also, transformation may be performedby using a coding unit as a transform block.

Splitting of a coding unit will be described in more detail withreference to FIGS. 3 through 16 . A current block and a neighboringblock according to the present disclosure may indicate one of a codingtree unit, a coding unit, a prediction block, and a transform block.Also, a current block or a current coding unit is a block on whichcurrently decoding or encoding is performed or a block on whichcurrently splitting is performed. A neighboring block may be a blockthat is reconstructed before a current block. A neighboring block may bespatially or temporally adjacent to a current block. A neighboring blockmay be located on bottom left, left, top left, top, top right, right,and bottom right of a current block.

FIG. 3 illustrates a process, performed by the image decoding apparatus100, of determining at least one coding unit by splitting a currentcoding unit, according to an embodiment.

A block shape may include 4N×4N, 4N×2N, 2N×4N, 4N×N, N×4N, 32N×N, N×32N,16N×N, N×16N, 8N×N or N×8N. Here, N may be a positive integer. Blockshape information is information indicating at least one of a shape, adirection, a ratio between a width and a height, and a size of a codingunit.

A shape of a coding unit may include a square and a non-square. Whenlengths of a width and a height of a coding unit are equal (that is,when a block shape of the coding unit is 4N×4N), the image decodingapparatus 100 may determine block shape information of the coding unitto be a square. The image decoding apparatus 100 may determine a shapeof a coding unit to be a non-square.

When a width and a height of a coding unit have different lengths (thatis, when a block shape of the coding unit is 4N×2N, 2N×4N, 4N×N, N×4N,32N×N, N×32N, 16N×N, N×16N, 8N×N, or N×8N), the image decoding apparatus100 may determine block shape information of the coding unit to be anon-square. When a shape of a coding unit is non-square, the imagedecoding apparatus 100 may determine a ratio of a width and a height ofblock shape information of the coding unit to be at least one of 1:2,2:1, 1:4, 4:1, 1:8, 8:1, 1:16, 16:1, 1:32, and 32:1. In addition, theimage decoding apparatus 100 may determine whether the coding unit is ina horizontal or vertical direction based on the lengths of the width andthe height of the coding unit. In addition, the image decoding apparatus100 may determine a size of the coding unit based on at least one of thelength of the width, the length of the height, or the area of the codingunit.

According to an embodiment, the image decoding apparatus 100 maydetermine a shape of a coding unit by using block shape information, andmay determine a splitting method of the coding unit by using split shapemode information. That is, a coding unit splitting method indicated bythe split shape mode information may be determined based on a blockshape indicated by the block shape information used by the imagedecoding apparatus 100.

The image decoding apparatus 100 may obtain split shape mode informationfrom a bitstream. However, the present disclosure is not limitedthereto, and the image decoding apparatus 100 and the image encodingapparatus 150 may determine previously agreed split shape modeinformation based on block shape information. The image decodingapparatus 100 may determine previously agreed split shape modeinformation with respect to a coding tree unit or a minimum coding unit.For example, the image decoding apparatus 100 may determine split shapemode information with respect to a coding tree unit, as quad split.Also, the image decoding apparatus 100 may determine split shape modeinformation with respect to a minimum coding unit, as “no split.” Indetail, the image decoding apparatus 100 may determine a size of acoding tree unit to be 256×256. The image decoding apparatus 100 maydetermine previously agreed split shape mode information as quad-split.Quad-split is a split shape mode in which a width and a height of acoding unit are both split into two. The image decoding apparatus 100may obtain a coding unit having a 128×128 size from a coding tree unithaving a 256×256 size based on split shape mode information. Inaddition, the image decoding apparatus 100 may determine a size of aminimum coding unit to be 4×4. The image decoding apparatus 100 mayobtain split shape mode information indicating “no split” with respectto a minimum coding unit.

According to an embodiment, the image decoding apparatus 100 may useblock shape information indicating that a current coding unit has asquare shape. For example, the image decoding apparatus 100 maydetermine whether not to split a square coding unit, whether tovertically split the square coding unit, whether to horizontally splitthe square coding unit, or whether to split the square coding unit intofour coding units, based on the split shape mode information. Referringto FIG. 3 , when the block shape information of a current coding unit300 indicates a square shape, an image decoder 115 may determine that acoding unit 310 a having the same size as the current coding unit 300 isnot split, based on the split shape mode information indicating not toperform splitting, or may determine coding units 310 b, 310 c, 310 d,310 e, and 310 f split based on the split shape mode informationindicating a predetermined splitting method.

Referring to FIG. 3 , according to an embodiment, the image decodingapparatus 100 may determine two coding units 310 b obtained by splittingthe current coding unit 300 in a vertical direction, based on the splitshape mode information indicating to perform splitting in a verticaldirection. The image decoding apparatus 100 may determine two codingunits 310 c obtained by splitting the current coding unit 300 in ahorizontal direction, based on the split shape mode informationindicating to perform splitting in a horizontal direction. The imagedecoding apparatus 100 may determine four coding units 310 d obtained bysplitting the current coding unit 300 in vertical and horizontaldirections, based on the split shape mode information indicating toperform splitting in vertical and horizontal directions. According to anembodiment, the image decoding apparatus 100 may determine three codingunits 310 e obtained by splitting the current coding unit 300 in avertical direction, based on the split shape mode information indicatingto perform ternary-splitting in a vertical direction. The image decodingapparatus 100 may determine three coding units 310 f obtained bysplitting the current coding unit 300 in a horizontal direction, basedon the split shape mode information indicating to performternary-splitting in a horizontal direction. However, splitting methodsof the square coding unit are not limited to the above-describedmethods, and the split shape mode information may indicate variousmethods. Predetermined splitting methods of splitting the square codingunit will be described in detail below in relation to variousembodiments.

FIG. 4 illustrates a process, performed by the image decoding apparatus100, of determining at least one coding unit by splitting a non-squarecurrent coding unit, according to an embodiment.

According to an embodiment, the image decoding apparatus 100 may useblock shape information indicating that a current coding unit has anon-square shape. The image decoding apparatus 100 may determine whethernot to split the non-square current coding unit or whether to split thenon-square current coding unit by using a predetermined splittingmethod, based on split shape mode information. Referring to FIG. 11 ,when block shape information of a current coding unit 400 or 450indicates a non-square shape, the image decoding apparatus 100 maydetermine a coding unit 410 or 460 having the same size as the currentcoding unit 400 or 450, based on split shape mode information indicatingnot to perform splitting, or determine coding units 420 a, 420 b, 430 ato 430 c, 470 a, 470 b, or 480 a to 480 c split based on split shapemode information indicating a predetermined splitting method.Predetermined splitting methods of splitting a non-square coding unitwill be described in detail below in relation to various embodiments.

According to an embodiment, the image decoding apparatus 100 maydetermine a splitting method of a coding unit by using the split shapemode information, and in this case, the split shape mode information mayindicate the number of one or more coding units generated by splitting acoding unit. Referring to FIG. 4 , when the split shape mode informationindicates to split the current coding unit 400 or 450 into two codingunits, the image decoding apparatus 100 may determine two coding units420 a and 420 b, or 470 a and 470 b included in the current coding unit400 or 450, by splitting the current coding unit 400 or 450 based on thesplit shape mode information.

According to an embodiment, when the image decoding apparatus 100 splitsthe non-square current coding unit 400 or 450 based on the split shapemode information, the location of a long side of the non-square currentcoding unit 400 or 450 may be considered. For example, the imagedecoding apparatus 100 may determine a plurality of coding units bysplitting a long side of the current coding unit 400 or 450, inconsideration of the shape of the current coding unit 400 or 450.

According to an embodiment, when the split shape mode informationindicates to split a coding unit into an odd number of blocks, the imagedecoding apparatus 100 may determine an odd number of coding unitsincluded in the current coding unit 400 or 450. For example, when thesplit shape mode information indicates to split the current coding unit400 or 450 into three coding units, the image decoding apparatus 100 maysplit the current coding unit 400 or 450 into three coding units 430 a,430 b, and 430 c, or 480 a, 480 b, and 480 c.

According to an embodiment, a ratio between a width and a height of thecurrent coding unit 400 or 450 may be 4:1 or 1:4. When the ratio betweenthe width and the height is 4:1, because a length of the width isgreater than a length of the height, the block shape information may bea horizontal direction. When the ratio between the width and the heightis 1:4, because a length of the width is shorter than a length of theheight, the block shape information may be a vertical direction. Theimage decoding apparatus 100 may determine to split a current codingunit into an odd number of blocks based on the split shape modeinformation. Also, the image decoding apparatus 100 may determine asplitting direction of the current coding unit 400 or 450 based on theblock shape information of the current coding unit 400 or 450. Forexample, when the current coding unit 400 is in a vertical direction,the image decoding apparatus 100 may determine coding units 430 a, 430b, and 430 c by splitting the current coding unit 400 in a horizontaldirection. For example, when the current coding unit 450 is in ahorizontal direction, the image decoding apparatus 100 may determinecoding units 480 a, 480 b, and 480 c by splitting the current codingunit 450 in a vertical direction.

According to an embodiment, the image decoding apparatus 100 maydetermine an odd number of coding units included in the current codingunit 400 or 450, and not all the determined coding units may have thesame size. For example, a predetermined coding unit 430 b or 480 b fromamong the determined odd number of coding units 430 a, 430 b, and 430 c,or 480 a, 480 b, and 480 c may have a size different from the size ofthe other coding units 430 a and 430 c, or 480 a and 480 c. That is,coding units which may be determined by splitting the current codingunit 400 or 450 may have multiple sizes, and in some cases, all of theodd number of coding units 430 a, 430 b, and 430 c, or 480 a, 480 b, and480 c may have different sizes.

According to an embodiment, when the split shape mode informationindicates to split a coding unit into an odd number of blocks, the imagedecoding apparatus 100 may determine an odd number of coding unitsincluded in the current coding unit 400 or 450, and may put apredetermined restriction on at least one coding unit from among the oddnumber of coding units generated by splitting the coding unit. Referringto FIG. 4 , the image decoding apparatus 100 may allow a decoding methodof the coding unit 430 b or 480 b to be different from that of the othercoding units 430 a and 430 c, or 480 a and 480 c, wherein the codingunit 430 b or 480 b is at a center location from among the three codingunits 430 a, 430 b, and 430 c, or 480 a, 480 b, and 480 c generated bysplitting the current coding unit 400 or 450. For example, the imagedecoding apparatus 100 may restrict the coding unit 430 b or 480 b atthe center location to be no longer split or to be split only apredetermined number of times, unlike the other coding units 430 a and430 c, or 480 a and 480 c.

FIG. 5 illustrates a process, performed by the image decoding apparatus100, of splitting a coding unit based on at least one of block shapeinformation and split shape mode information, according to anembodiment.

According to an embodiment, the image decoding apparatus 100 maydetermine to split or not to split a square first coding unit 500 intocoding units, based on at least one of the block shape information andthe split shape mode information. According to an embodiment, when thesplit shape mode information indicates to split the first coding unit500 in a horizontal direction, the image decoding apparatus 100 maydetermine a second coding unit 510 by splitting the first coding unit500 in a horizontal direction. A first coding unit, a second codingunit, and a third coding unit used according to an embodiment are termsused to understand a relation before and after splitting a coding unit.For example, a second coding unit may be determined by splitting a firstcoding unit, and a third coding unit may be determined by splitting thesecond coding unit. It will be understood that the structure of thefirst coding unit, the second coding unit, and the third coding unitfollows the above descriptions.

According to an embodiment, the image decoding apparatus 100 maydetermine to split or not to split the determined second coding unit 510into coding units, based on the split shape mode information. Referringto FIG. 5 , the image decoding apparatus 100 may split the non-squaresecond coding unit 510, which is determined by splitting the firstcoding unit 500, into one or more third coding units 520 a, or 520 b,520 c, and 520 d based on the split shape mode information, or may notsplit the non-square second coding unit 510. The image decodingapparatus 100 may obtain the split shape mode information, and determinea plurality of various-shaped second coding units (e.g., 510) bysplitting the first coding unit 500, based on the obtained split shapemode information, and the second coding unit 510 may be split by usingthe splitting method of the first coding unit 500, based on the splitshape mode information. According to an embodiment, when the firstcoding unit 500 is split into the second coding units 510 based on thesplit shape mode information of the first coding unit 500, the secondcoding unit 510 may also be split into the third coding units 520 a, or520 b, 520 c, and 520 d based on the split shape mode information of thesecond coding unit 510. That is, a coding unit may be recursively splitbased on the split shape mode information of each coding unit.Therefore, a square coding unit may be determined by splitting anon-square coding unit, and a non-square coding unit may be determinedby recursively splitting the square coding unit.

Referring to FIG. 5 , a predetermined coding unit from among an oddnumber of third coding units 520 b, 520 c, and 520 d determined bysplitting the non-square second coding unit 510 (e.g., a coding unitwhich is located at a center location or a square coding unit) may berecursively split. According to an embodiment, the square third codingunit 520 b from among the odd number of third coding units 520 b, 520 c,and 520 d may be split in a horizontal direction into a plurality offourth coding units. A non-square fourth coding unit 530 b or 530 d fromamong the plurality of fourth coding units 530 a, 530 b, 530 c, and 530d may be split into a plurality of coding units. For example, thenon-square fourth coding unit 530 d or 530 d may be split into an oddnumber of coding units. A method that may be used to recursively split acoding unit will be described below in relation to various embodiments.

According to an embodiment, the image decoding apparatus 100 maydetermine to split each of the third coding units 520 a, 520 b, 520 c,and 520 d into coding units based on the split shape mode information.Also, the image decoding apparatus 100 may determine not to split thesecond coding unit 510, based on the split shape mode information.According to an embodiment, the image decoding apparatus 100 may splitthe non-square second coding unit 510 into the odd number of thirdcoding units 520 b, 520 c, and 520 d. The image decoding apparatus 100may put a predetermined restriction on a predetermined third coding unitfrom among the odd number of third coding units 520 b, 520 c, and 520 d.For example, the image decoding apparatus 100 may restrict the thirdcoding unit 520 c at a center location from among the odd number ofthird coding units 520 b, 520 c, and 520 d to be no longer split or tobe split a settable number of times.

Referring to FIG. 5 , the image decoding apparatus 100 may restrict thethird coding unit 520 c, which is at the center location from among theodd number of third coding units 520 b, 520 c, and 520 d included in thenon-square second coding unit 510, to be no longer split, to be split byusing a predetermined splitting method (e.g., split into only fourcoding units or split by using a splitting method of the second codingunit 510), or to be split only a predetermined number of times (e.g.,split only n times (where n>0)). However, the restrictions on the thirdcoding unit 520 c at the center location are not limited to theabove-described examples, and may include various restrictions fordecoding the third coding unit 520 c at the center location differentlyfrom the other third coding units 520 b and 520 d.

According to an embodiment, the image decoding apparatus 100 may obtainthe split shape mode information, which is used to split a currentcoding unit, from a predetermined location in the current coding unit.

FIG. 6 illustrates a method, performed by the image decoding apparatus100, of determining a predetermined coding unit from among an odd numberof coding units, according to an embodiment.

Referring to FIG. 6 , split shape mode information of a current codingunit 600 or 650 may be obtained from a sample of a predeterminedlocation from among a plurality of samples included in the currentcoding unit 600 or 650 (e.g., a sample 640 or 690 of a center location).However, the predetermined location in the current coding unit 600, fromwhich at least one of the split shape mode information may be obtained,is not limited to the center location in FIG. 6 , and may includevarious locations included in the current coding unit 600 (e.g., top,bottom, left, right, top left, bottom left, top right, and bottom rightlocations). The image decoding apparatus 100 may obtain the split shapemode information from the predetermined location and determine to splitor not to split the current coding unit into various-shaped andvarious-sized coding units.

According to an embodiment, when the current coding unit is split into apredetermined number of coding units, the image decoding apparatus 100may select one of the coding units. Various methods may be used toselect one of a plurality of coding units, as will be described below inrelation to various embodiments.

According to an embodiment, the image decoding apparatus 100 may splitthe current coding unit into a plurality of coding units, and maydetermine a coding unit at a predetermined location.

According to an embodiment, the image decoding apparatus 100 may useinformation indicating locations of the odd number of coding units, todetermine a coding unit at a center location from among the odd numberof coding units. Referring to FIG. 6 , the image decoding apparatus 100may determine an odd number of coding units 620 a, 620 b, and 620 c oran odd number of coding units 660 a, 660 b, and 660 c by splitting thecurrent coding unit 600 or the current coding unit 650. The imagedecoding apparatus 100 may determine a coding unit 620 b at a centerlocation or a coding unit 660 b at a center location by usinginformation about locations of the odd number of coding units 620 a to620 c or the odd number of coding units 660 a to 660 c. For example, theimage decoding apparatus 100 may determine the coding unit 620 b of thecenter location by determining the locations of the coding units 620 a,620 b, and 620 c based on information indicating locations ofpredetermined samples included in the coding units 620 a, 620 b, and 620c. In detail, the image decoding apparatus 100 may determine the codingunit 620 b at the center location by determining the locations of thecoding units 620 a, 620 b, and 620 c based on information indicatinglocations of top left samples 630 a, 630 b, and 630 c of the codingunits 620 a, 620 b, and 620 c.

According to an embodiment, the information indicating the locations ofthe top left samples 630 a, 630 b, and 630 c, which are included in thecoding units 620 a, 620 b, and 620 c, respectively, may includeinformation about locations or coordinates of the coding units 620 a,620 b, and 620 c in a picture. According to an embodiment, theinformation indicating the locations of the top left samples 630 a, 630b, and 630 c, which are included in the coding units 620 a, 620 b, and620 c, respectively, may include information indicating widths orheights of the coding units 620 a, 620 b, and 620 c included in thecurrent coding unit 600, and the widths or heights may correspond toinformation indicating differences between the coordinates of the codingunits 620 a, 620 b, and 620 c in the picture. That is, the imagedecoding apparatus 100 may determine the coding unit 620 b at the centerlocation by directly using the information about the locations orcoordinates of the coding units 620 a, 620 b, and 620 c in the picture,or by using the information about the widths or heights of the codingunits, which correspond to the difference values between thecoordinates.

According to an embodiment, information indicating the location of thetop left sample 630 a of the upper coding unit 620 a may includecoordinates (xa, ya), information indicating the location of the topleft sample 630 b of the middle coding unit 620 b may includecoordinates (xb, yb), and information indicating the location of the topleft sample 630 c of the lower coding unit 620 c may include coordinates(xc, yc). The image decoding apparatus 100 may determine the middlecoding unit 620 b by using the coordinates of the top left samples 630a, 630 b, and 630 c which are included in the coding units 620 a, 620 b,and 620 c, respectively. For example, when the coordinates of the topleft samples 630 a, 630 b, and 630 c are sorted in an ascending ordescending order, the coding unit 620 b including the coordinates (xb,yb) of the sample 630 b at a center location may be determined as acoding unit at a center location from among the coding units 620 a, 620b, and 620 c determined by splitting the current coding unit 600.However, the coordinates indicating the locations of the top leftsamples 630 a, 630 b, and 630 c may include coordinates indicatingabsolute locations in the picture, or may use coordinates (dxb, dyb)indicating a relative location of the top left sample 630 b of themiddle coding unit 620 b and coordinates (dxc, dyc) indicating arelative location of the top left sample 630 c of the lower coding unit620 c with reference to the location of the top left sample 630 a of theupper coding unit 620 a. A method of determining a coding unit at apredetermined location by using coordinates of a sample included in thecoding unit, as information indicating a location of the sample, is notlimited to the above-described method, and may include variousarithmetic methods capable of using the coordinates of the sample.

According to an embodiment, the image decoding apparatus 100 may splitthe current coding unit 600 into a plurality of coding units 620 a, 620b, and 620 c, and may select one of the coding units 620 a, 620 b, and620 c based on a predetermined criterion. For example, the imagedecoding apparatus 100 may select the coding unit 620 b, which has asize different from that of the others, from among the coding units 620a, 620 b, and 620 c.

According to an embodiment, the image decoding apparatus 100 maydetermine the widths or heights of the coding units 620 a, 620 b, and620 c by using the coordinates (xa, ya) indicating the location of thetop left sample 630 a of the upper coding unit 620 a, the coordinates(xb, yb) indicating the location of the top left sample 630 b of themiddle coding unit 620 b, and the coordinates (xc, yc) indicating thelocation of the top left sample 630 c of the lower coding unit 620 c.The image decoding apparatus 100 may determine the respective sizes ofthe coding units 620 a, 620 b, and 620 c by using the coordinates (xa,ya), (xb, yb), and (xc, yc) indicating the locations of the coding units620 a, 620 b, and 620 c. According to an embodiment, the image decodingapparatus 100 may determine the width of the upper coding unit 620 a tobe a width of the current coding unit 600. According to an embodiment,the image decoding apparatus 100 may determine the height of the uppercoding unit 620 a to be yb-ya. According to an embodiment, the imagedecoding apparatus 100 may determine a width of the middle coding unit620 b to be the width of the current coding unit 600. According to anembodiment, the image decoding apparatus 100 may determine the height ofthe middle coding unit 620 b to be yc-yb. According to an embodiment,the image decoding apparatus 100 may determine the width or height ofthe lower coding unit 620 c by using the width or height of the currentcoding unit 600 or the widths or heights of the upper and middle codingunits 620 a and 620 b. The image decoding apparatus 100 may determine acoding unit, which has a size different from that of the others, basedon the determined widths and heights of the coding units 620 a to 620 c.Referring to FIG. 6 , the image decoding apparatus 100 may determine themiddle coding unit 620 b, which has a size different from the size ofthe upper and lower coding units 620 a and 620 c, as the coding unit ofthe predetermined location. However, the above-described method,performed by the image decoding apparatus 100, of determining a codingunit having a size different from the size of the other coding unitsmerely corresponds to an example of determining a coding unit at apredetermined location by using the sizes of coding units, which aredetermined based on coordinates of samples, and thus various methods ofdetermining a coding unit at a predetermined location by comparing thesizes of coding units, which are determined based on coordinates ofpredetermined samples, may be used.

The image decoding apparatus 100 may determine the widths or heights ofthe coding units 660 a, 660 b, and 660 c by using the coordinates (xd,yd) indicating a location of a top left sample 670 a of the left codingunit 660 a, the coordinates (xe, ye) indicating a location of a top leftsample 670 b of the middle coding unit 660 b, and the coordinates (xf,yf) indicating a location of a top left sample 670 c of the right codingunit 660 c. The image decoding apparatus 100 may determine therespective sizes of the coding units 660 a, 660 b, and 660 c by usingthe coordinates (xd, yd), (xe, ye), and (xf, yf) indicating thelocations of the coding units 660 a, 660 b, and 660 c.

According to an embodiment, the image decoding apparatus 100 maydetermine the width of the left coding unit 660 a to be xe-xd. The imagedecoding apparatus 100 may determine a height of the left coding unit660 a to be a height of the current coding unit 650. According to anembodiment, the image decoding apparatus 100 may determine the width ofthe middle coding unit 660 b to be xf-xe. The image decoding apparatus100 may determine a height of the middle coding unit 660 b to be theheight of the current coding unit 600. According to an embodiment, theimage decoding apparatus 100 may determine the width or height of theright coding unit 660 c by using the width or height of the currentcoding unit 650 or the widths or heights of the left and middle codingunits 660 a and 660 b. The image decoding apparatus 100 may determine acoding unit, which has a size different from that of the others, basedon the determined widths and heights of the coding units 660 a to 660 c.Referring to FIG. 6 , the image decoding apparatus 100 may determine themiddle coding unit 660 b, which has a size different from the size ofthe left and right coding units 660 a and 660 c, as the coding unit ofthe predetermined location. However, the above-described method,performed by the image decoding apparatus 100, of determining a codingunit having a size different from the size of the other coding unitsmerely corresponds to an example of determining a coding unit at apredetermined location by using the sizes of coding units, which aredetermined based on coordinates of samples, and thus various methods ofdetermining a coding unit at a predetermined location by comparing thesizes of coding units, which are determined based on coordinates ofpredetermined samples, may be used.

However, locations of samples considered to determine locations ofcoding units are not limited to the above-described top left locations,and information about arbitrary locations of samples included in thecoding units may be used.

According to an embodiment, the image decoding apparatus 100 may selecta coding unit at a predetermined location from among an odd number ofcoding units determined by splitting the current coding unit,considering the shape of the current coding unit. For example, when thecurrent coding unit has a non-square shape, a width of which is longerthan a height, the image decoding apparatus 100 may determine the codingunit at the predetermined location in a horizontal direction. That is,the image decoding apparatus 100 may determine one of coding units atdifferent locations in a horizontal direction and put a restriction onthe coding unit. When the current coding unit has a non-square shape, aheight of which is longer than a width, the image decoding apparatus 100may determine the coding unit at the predetermined location in avertical direction. That is, the image decoding apparatus 100 maydetermine one of coding units at different locations in a verticaldirection and may put a restriction on the coding unit.

According to an embodiment, the image decoding apparatus 100 may useinformation indicating respective locations of an even number of codingunits, to determine the coding unit at the predetermined location fromamong the even number of coding units. The image decoding apparatus 100may determine an even number of coding units by splitting (binary split)the current coding unit, and may determine the coding unit at thepredetermined location by using the information about the locations ofthe even number of coding units. An operation related thereto maycorrespond to the operation of determining a coding unit at apredetermined location (e.g., a center location) from among an oddnumber of coding units, which has been described in detail above inrelation to FIG. 6 , and thus detailed descriptions thereof are notprovided here.

According to an embodiment, when a non-square current coding unit issplit into a plurality of coding units, predetermined information abouta coding unit at a predetermined location may be used in a splittingoperation to determine the coding unit at the predetermined locationfrom among the plurality of coding units. For example, the imagedecoding apparatus 100 may use at least one of block shape informationand split shape mode information, which is stored in a sample includedin a coding unit at a center location, in a splitting operation todetermine the coding unit at the center location from among theplurality of coding units determined by splitting the current codingunit.

Referring to FIG. 6 , the image decoding apparatus 100 may split thecurrent coding unit 600 into a plurality of coding units 620 a, 620 b,and 620 c based on the split shape mode information, and may determinethe coding unit 620 b at the center location from among the plurality ofthe coding units 620 a, 620 b, and 620 c. Furthermore, the imagedecoding apparatus 100 may determine the coding unit 620 b at the centerlocation, in consideration of a location from which the split shape modeinformation is obtained. That is, the split shape mode information ofthe current coding unit 600 may be obtained from the sample 640 at acenter location of the current coding unit 600, and when the currentcoding unit 600 is split into the plurality of coding units 620 a, 620b, and 620 c based on the split shape mode information, the coding unit620 b including the sample 640 may be determined as the coding unit atthe center location. However, information used to determine the codingunit at the center location is not limited to the split shape modeinformation, and various types of information may be used to determinethe coding unit at the center location.

According to an embodiment, predetermined information for identifyingthe coding unit at the predetermined location may be obtained from apredetermined sample included in a coding unit to be determined.Referring to FIG. 6 , the image decoding apparatus 100 may use the splitshape mode information, which is obtained from a sample at apredetermined location in the current coding unit 600 (e.g., a sample ata center location of the current coding unit 600) to determine a codingunit at a predetermined location from among the plurality of the codingunits 620 a, 620 b, and 620 c determined by splitting the current codingunit 600 (e.g., a coding unit at a center location from among aplurality of split coding units). That is, the image decoding apparatus100 may determine the sample at the predetermined location byconsidering a block shape of the current coding unit 600, determine thecoding unit 620 b including a sample, from which predeterminedinformation (e.g., the split shape mode information) may be obtained,from among the plurality of coding units 620 a, 620 b, and 620 cdetermined by splitting the current coding unit 600, and may put apredetermined restriction on the coding unit 620 b. Referring to FIG. 6, according to an embodiment, the image decoding apparatus 100 maydetermine the sample 640 at the center location of the current codingunit 600 as the sample from which the predetermined information may beobtained, and may put a predetermined restriction on the coding unit 620b including the sample 640, in a decoding operation. However, thelocation of the sample from which the predetermined information may beobtained is not limited to the above-described location, and may includearbitrary locations of samples included in the coding unit 620 b to bedetermined for a restriction.

According to an embodiment, the location of the sample from which thepredetermined information may be obtained may be determined based on theshape of the current coding unit 600. According to an embodiment, theblock shape information may indicate whether the current coding unit hasa square or non-square shape, and the location of the sample from whichthe predetermined information may be obtained may be determined based onthe shape. For example, the image decoding apparatus 100 may determine asample located on a boundary for dividing at least one of a width andheight of the current coding unit in half, as the sample from which thepredetermined information may be obtained, by using at least one ofinformation about the width of the current coding unit and informationabout the height of the current coding unit. As another example, whenthe block shape information of the current coding unit indicates anon-square shape, the image decoding apparatus 100 may determine one ofsamples adjacent to a boundary for dividing a long side of the currentcoding unit in half, as the sample from which the predeterminedinformation may be obtained.

According to an embodiment, when the current coding unit is split into aplurality of coding units, the image decoding apparatus 100 may use thesplit shape mode information to determine a coding unit at apredetermined location from among the plurality of coding units.According to an embodiment, the image decoding apparatus 100 may obtainthe split shape mode information from a sample at a predeterminedlocation in a coding unit, and split the plurality of coding units,which are generated by splitting the current coding unit, by using thesplit shape mode information, which is obtained from the sample of thepredetermined location in each of the plurality of coding units. Thatis, a coding unit may be recursively split based on the split shape modeinformation, which is obtained from the sample at the predeterminedlocation in each coding unit. An operation of recursively splitting acoding unit has been described above in relation to FIG. 5 , and thusdetailed descriptions thereof will not be provided here.

According to an embodiment, the image decoding apparatus 100 maydetermine one or more coding units by splitting the current coding unit,and may determine an order of decoding the one or more coding units,based on a predetermined block (e.g., the current coding unit).

FIG. 7 illustrates an order of processing a plurality of coding unitswhen the image decoding apparatus 100 determines the plurality of codingunits by splitting a current coding unit, according to an embodiment.

According to an embodiment, the image decoding apparatus 100 maydetermine second coding units 710 a and 710 b by splitting a firstcoding unit 700 in a vertical direction, determine second coding units730 a and 730 b by splitting the first coding unit 700 in a horizontaldirection, or determine second coding units 750 a to 750 d by splittingthe first coding unit 700 in vertical and horizontal directions, basedon split shape mode information.

Referring to FIG. 7 , the image decoding apparatus 100 may determine toprocess the second coding units 710 a and 710 b, which are determined bysplitting the first coding unit 700 in a vertical direction, in ahorizontal direction order 710 c. The image decoding apparatus 100 maydetermine to process the second coding units 730 a and 730 b, which aredetermined by splitting the first coding unit 700 in a horizontaldirection, in a vertical direction order 730 c. The image decodingapparatus 100 may determine to process the second coding units 750 a to750 d, which are determined by splitting the first coding unit 700 invertical and horizontal directions, according to a predetermined order(e.g., in a raster scan order or Z-scan order 750 e) by which codingunits in a row are processed and then coding units in a next row areprocessed.

According to an embodiment, the image decoding apparatus 100 mayrecursively split coding units. Referring to FIG. 7 , the image decodingapparatus 100 may determine a plurality of coding units 710 a, 710 b,730 a, 730 b, 750 a, 750 b, 750 c, and 750 d by splitting the firstcoding unit 700, and may recursively split each of the determinedplurality of coding units 710 a, 710 b, 730 a, 730 b, 750 a, 750 b, 750c, and 750 d. A splitting method of the plurality of coding units 710 a,710 b, 730 a, 730 b, 750 a, 750 b, 750 c, and 750 d may correspond to asplitting method of the first coding unit 700. As such, each of theplurality of coding units 710 a, 710 b, 730 a, 730 b, 750 a, 750 b, 750c, and 750 d may be independently split into a plurality of codingunits. Referring to FIG. 7 , the image decoding apparatus 100 maydetermine the second coding units 710 a and 710 b by splitting the firstcoding unit 700 in a vertical direction, and may determine toindependently split or not to split each of the second coding units 710a and 710 b.

According to an embodiment, the image decoding apparatus 100 maydetermine third coding units 720 a and 720 b by splitting the leftsecond coding unit 710 a in a horizontal direction, and may not splitthe right second coding unit 710 b.

According to an embodiment, a processing order of coding units may bedetermined based on an operation of splitting a coding unit. In otherwords, a processing order of split coding units may be determined basedon a processing order of coding units immediately before being split.The image decoding apparatus 100 may determine a processing order of thethird coding units 720 a and 720 b determined by splitting the leftsecond coding unit 710 a, independently of the right second coding unit710 b. Because the third coding units 720 a and 720 b are determined bysplitting the left second coding unit 710 a in a horizontal direction,the third coding units 720 a and 720 b may be processed in a verticaldirection order 720 c. Because the left and right second coding units710 a and 710 b are processed in the horizontal direction order 710 c,the right second coding unit 710 b may be processed after the thirdcoding units 720 a and 720 b included in the left second coding unit 710a are processed in the vertical direction order 720 c. An operation ofdetermining a processing order of coding units based on a coding unitbefore being split is not limited to the above-described example, andvarious methods may be used to independently process coding units, whichare split and determined to various shapes, in a predetermined order.

FIG. 8 illustrates a process, performed by the image decoding apparatus100, of determining that a current coding unit is to be split into anodd number of coding units, when the coding units are not processable ina predetermined order, according to an embodiment.

According to an embodiment, the image decoding apparatus 100 maydetermine whether the current coding unit is split into an odd number ofcoding units, based on obtained split shape mode information. Referringto FIG. 8 , a square first coding unit 800 may be split into non-squaresecond coding units 810 a and 810 b, and the second coding units 810 aand 810 b may be independently split into third coding units 820 a and820 b, and 820 c to 820 e. According to an embodiment, the imagedecoding apparatus 100 may determine a plurality of third coding units820 a and 820 b by splitting the left second coding unit 810 a in ahorizontal direction, and may split the right second coding unit 810 binto an odd number of third coding units 820 c to 820 e.

According to an embodiment, the image decoding apparatus 100 maydetermine whether any coding unit is split into an odd number of codingunits, by determining whether the third coding units 820 a and 820 b,and 820 c to 820 e are processable in a predetermined order. Referringto FIG. 8 , the image decoding apparatus 100 may determine the thirdcoding units 820 a and 820 b, and 820 c to 820 e by recursivelysplitting the first coding unit 800. The image decoding apparatus 100may determine whether any of the first coding unit 800, the secondcoding units 810 a and 810 b, and the third coding units 820 a and 820b, and 820 c, 820 d, and 820 e are split into an odd number of codingunits, based on at least one of block shape information and the splitshape mode information. For example, a second coding unit located in theright from among the second coding units 810 a and 810 b may be splitinto an odd number of third coding units 820 c, 820 d, and 820 e. Aprocessing order of a plurality of coding units included in the firstcoding unit 800 may be a predetermined order (e.g., a Z-scan order 830),and the image decoding apparatus 100 may decide whether the third codingunits 820 c, 820 d, and 820 e, which are determined by splitting theright second coding unit 810 b into an odd number of coding units,satisfy a condition for processing in the predetermined order.

According to an embodiment, the image decoding apparatus 100 maydetermine whether the third coding units 820 a and 820 b, and 820 c, 820d, and 820 e included in the first coding unit 800 satisfy the conditionfor processing in the predetermined order, and the condition relates towhether at least one of a width and height of the second coding units810 a and 810 b is to be split in half along a boundary of the thirdcoding units 820 a and 820 b, and 820 c, 820 d, and 820 e. For example,the third coding units 820 a and 820 b determined by dividing the heightof the non-square left second coding unit 810 a in half may satisfy thecondition. Because boundaries of the third coding units 820 c, 820 d,and 820 e determined by splitting the right second coding unit 810 binto three coding units do not divide the width or height of the rightsecond coding unit 810 b in half, it may be determined that the thirdcoding units 820 c, 820 d, and 820 e do not satisfy the condition. Whenthe condition is not satisfied as described above, the image decodingapparatus 100 may decide disconnection of a scan order, and determinethat the right second coding unit 810 b is split into an odd number ofcoding units, based on a result of the decision. According to anembodiment, when a coding unit is split into an odd number of codingunits, the image decoding apparatus 100 may put a predeterminedrestriction on a coding unit at a predetermined location among the splitcoding units. The restriction or the predetermined location has beendescribed above in relation to various embodiments, and thus detaileddescriptions thereof will not be provided here.

FIG. 9 illustrates a process, performed by the image decoding apparatus100, of determining at least one coding unit by splitting a first codingunit 900, according to an embodiment.

According to an embodiment, the image decoding apparatus 100 may splitthe first coding unit 900, based on split shape mode information, whichis obtained by a receiver (not shown). The square first coding unit 900may be split into four square coding units, or may be split into aplurality of non-square coding units. For example, referring to FIG. 9 ,when the first coding unit 900 has a square shape and the split shapemode information indicates to split the first coding unit 900 intonon-square coding units, the image decoding apparatus 100 may split thefirst coding unit 900 into a plurality of non-square coding units. Indetail, when the split shape mode information indicates to determine anodd number of coding units by splitting the first coding unit 900 in ahorizontal direction or a vertical direction, the image decodingapparatus 100 may split the square first coding unit 900 into an oddnumber of coding units, e.g., second coding units 910 a, 910 b, and 910c determined by splitting the square first coding unit 900 in a verticaldirection or second coding units 920 a, 920 b, and 920 c determined bysplitting the square first coding unit 900 in a horizontal direction.

According to an embodiment, the image decoding apparatus 100 maydetermine whether the second coding units 910 a, 910 b, 910 c, 920 a,920 b, and 920 c included in the first coding unit 900 satisfy acondition for processing in a predetermined order, and the conditionrelates to whether at least one of a width and height of the firstcoding unit 900 is to be divided in half along a boundary of the secondcoding units 910 a, 910 b, 910 c, 920 a, 920 b, and 920 c. Referring toFIG. 9 , because boundaries of the second coding units 910 a, 910 b, and910 c determined by splitting the square first coding unit 900 in avertical direction do not divide the height of the first coding unit 900in half, it may be determined that the first coding unit 900 does notsatisfy the condition for processing in the predetermined order. Inaddition, because boundaries of the second coding units 920 a, 920 b,and 920 c determined by splitting the square first coding unit 900 in ahorizontal direction do not divide the width of the first coding unit900 in half, it may be determined that the first coding unit 900 doesnot satisfy the condition for processing in the predetermined order.When the condition is not satisfied as described above, the imagedecoding apparatus 100 may decide disconnection of a scan order, and maydetermine that the first coding unit 900 is to be split into an oddnumber of coding units, based on a result of the decision. According toan embodiment, when a coding unit is split into an odd number of codingunits, the image decoding apparatus 100 may put a predeterminedrestriction on a coding unit at a predetermined location among the splitcoding units. The restriction or the predetermined location has beendescribed above in relation to various embodiments, and thus detaileddescriptions thereof will not be provided here.

According to an embodiment, the image decoding apparatus 100 maydetermine various-shaped coding units by splitting a first coding unit.

Referring to FIG. 9 , the image decoding apparatus 100 may split thesquare first coding unit 900 or a non-square first coding unit 930 or950 into various-shaped coding units.

FIG. 10 illustrates that a shape into which a second coding unit issplittable by the image decoding apparatus 100 is restricted when thesecond coding unit having a non-square shape, which is determined bysplitting a first coding unit 1000, satisfies a predetermined condition,according to an embodiment.

According to an embodiment, the image decoding apparatus 100 maydetermine to split the square first coding unit 1000 into non-squaresecond coding units 1010 a, 1010 b, 1020 a, and 1020 b, based on splitshape mode information, which is obtained by a receiver (not shown). Thesecond coding units 1010 a, 1010 b, 1020 a, and 1020 b may beindependently split. As such, the image decoding apparatus 100 maydetermine to split or not to split the first coding unit 1000 into aplurality of coding units, based on the split shape mode information ofeach of the second coding units 1010 a, 1010 b, 1020 a, and 1020 b.According to an embodiment, the image decoding apparatus 100 maydetermine third coding units 1012 a and 1012 b by splitting thenon-square left second coding unit 1010 a, which is determined bysplitting the first coding unit 1000 in a vertical direction, in ahorizontal direction. However, when the left second coding unit 1010 ais split in a horizontal direction, the image decoding apparatus 100 mayrestrict the right second coding unit 1010 b to not be split in ahorizontal direction in which the left second coding unit 1010 a issplit. When third coding units 1014 a and 1014 b are determined bysplitting the right second coding unit 1010 b in a same direction,because the left and right second coding units 1010 a and 1010 b areindependently split in a horizontal direction, the third coding units1012 a, 1012 b, 1014 a, and 1014 b may be determined. However, this caseserves equally as a case in which the image decoding apparatus 100splits the first coding unit 1000 into four square second coding units1030 a, 1030 b, 1030 c, and 1030 d, based on the split shape modeinformation, and may be inefficient in terms of image decoding.

According to an embodiment, the image decoding apparatus 100 maydetermine third coding units 1022 a, 1022 b, 1024 a, and 1024 b bysplitting the non-square second coding unit 1020 a or 1020 b, which isdetermined by splitting the first coding unit 1000 in a horizontaldirection, in a vertical direction. However, when a second coding unit(e.g., the upper second coding unit 1020 a) is split in a verticaldirection, for the above-described reason, the image decoding apparatus100 may restrict the other second coding unit (e.g., the lower secondcoding unit 1020 b) to not be split in a vertical direction in which theupper second coding unit 1020 a is split.

FIG. 11 illustrates a process, performed by the image decoding apparatus100, of splitting a square coding unit when split shape mode informationindicates that the square coding unit is not to be split into foursquare coding units, according to an embodiment.

According to an embodiment, the image decoding apparatus 100 maydetermine second coding units 1110 a, 1110 b, 1120 a, 1120 b, etc. bysplitting a first coding unit 1100, based on the split shape modeinformation. The split shape mode information may include informationabout various methods of splitting a coding unit, but the informationabout various splitting methods may not include information forsplitting a coding unit into four square coding units. According to suchsplit shape mode information, the image decoding apparatus 100 may notsplit the first square coding unit 1100 into four square second codingunits 1130 a, 1130 b, 1130 c, and 1130 d. The image decoding apparatus100 may determine the non-square second coding units 1110 a, 1110 b, 110a, 1120 b, etc., based on the split shape mode information.

According to an embodiment, the image decoding apparatus 100 mayindependently split the non-square second coding units 1110 a, 1110 b,1120 a, 1120 b, etc. Each of the second coding units 1110 a, 1110 b,1120 a, 1120 b, etc. may be recursively split in a predetermined order,and this splitting method may correspond to a method of splitting thefirst coding unit 1100, based on the split shape mode information.

For example, the image decoding apparatus 100 may determine square thirdcoding units 1112 a and 1112 b by splitting the left second coding unit1110 a in a horizontal direction, and may determine square third codingunits 1114 a and 1114 b by splitting the right second coding unit 1110 bin a horizontal direction. Furthermore, the image decoding apparatus 100may determine square third coding units 1116 a, 1116 b, 1116 c, and 1116d by splitting both of the left and right second coding units 1110 a and1110 b in a horizontal direction. In this case, coding units having thesame shape as the four square second coding units 1130 a, 1130 b, 1130c, and 1130 d split from the first coding unit 1100 may be determined.

As another example, the image decoding apparatus 100 may determinesquare third coding units 1122 a and 1122 b by splitting the uppersecond coding unit 1120 a in a vertical direction, and may determinesquare third coding units 1124 a and 1124 b by splitting the lowersecond coding unit 1120 b in a vertical direction. Furthermore, theimage decoding apparatus 100 may determine square third coding units1126 a, 1126 b, 1126 c, and 1126 d by splitting both of the upper andlower second coding units 1120 a and 1120 b in a vertical direction. Inthis case, coding units having the same shape as the four square secondcoding units 1130 a, 1130 b, 1130 c, and 1130 d split from the firstcoding unit 1100 may be determined.

FIG. 12 illustrates that a processing order between a plurality ofcoding units may be changed depending on a process of splitting a codingunit, according to an embodiment.

According to an embodiment, the image decoding apparatus 100 may split afirst coding unit 1200, based on split shape mode information. When ablock shape is a square shape and the split shape mode informationindicates to split the first coding unit 1200 in at least one ofhorizontal and vertical directions, the image decoding apparatus 100 maydetermine second coding units 1210 a, 1210 b, 1220 a, 1220 b, etc. bysplitting the first coding unit 1200. Referring to FIG. 12 , thenon-square second coding units 1210 a, 1210 b, 1220 a, and 1220 bdetermined by splitting the first coding unit 1200 in only a horizontaldirection or vertical direction may be independently split based on thesplit shape mode information of each coding unit. For example, the imagedecoding apparatus 100 may determine third coding units 1216 a, 1216 b,1216 c, and 1216 d by splitting the second coding units 1210 a and 1210b, which are generated by splitting the first coding unit 1200 in avertical direction, in a horizontal direction, and may determine thirdcoding units 1226 a, 1226 b, 1226 c, and 1226 d by splitting the secondcoding units 1220 a and 1220 b, which are generated by splitting thefirst coding unit 1200 in a horizontal direction, in a verticaldirection. An operation of splitting the second coding units 1210 a,1210 b, 1220 a, and 1220 b has been described above in relation to FIG.11 , and thus detailed descriptions thereof will not be provided herein.

According to an embodiment, the image decoding apparatus 100 may processcoding units in a predetermined order. An operation of processing codingunits in a predetermined order has been described above in relation toFIG. 7 , and thus detailed descriptions thereof will not be providedherein. Referring to FIG. 12 , the image decoding apparatus 100 maydetermine four square third coding units 1216 a, 1216 b, 1216 c, and1216 d, and 1226 a, 1226 b, 1226 c, and 1226 d by splitting the squarefirst coding unit 1200. According to an embodiment, the image decodingapparatus 100 may determine processing orders of the third coding units1216 a, 1216 b, 1216 c, and 1216 d, and 1226 a, 1226 b, 1226 c, and 1226d based on a splitting method of the first coding unit 1200.

According to an embodiment, the image decoding apparatus 100 maydetermine the third coding units 1216 a, 1216 b, 1216 c, and 1216 d bysplitting the second coding units 1210 a and 1210 b generated bysplitting the first coding unit 1200 in a vertical direction, in ahorizontal direction, and may process the third coding units 1216 a,1216 b, 1216 c, and 1216 d in a processing order 1217 for initiallyprocessing the third coding units 1216 a and 1216 c, which are includedin the left second coding unit 1210 a, in a vertical direction, and thenprocessing the third coding unit 1216 b and 1216 d, which are includedin the right second coding unit 1210 b, in a vertical direction.

According to an embodiment, the image decoding apparatus 100 maydetermine the third coding units 1226 a, 1226 b, 1226 c, and 1226 d bysplitting the second coding units 1220 a and 1220 b generated bysplitting the first coding unit 1200 in a horizontal direction, in avertical direction, and may process the third coding units 1226 a, 1226b, 1226 c, and 1226 d in a processing order 1227 for initiallyprocessing the third coding units 1226 a and 1226 b, which are includedin the upper second coding unit 1220 a, in a horizontal direction, andthen processing the third coding unit 1226 c and 1226 d, which areincluded in the lower second coding unit 1220 b, in a horizontaldirection.

Referring to FIG. 12 , the square third coding units 1216 a, 1216 b,1216 c, and 1216 d, and 1226 a, 1226 b, 1226 c, and 1226 d may bedetermined by splitting the second coding units 1210 a, 1210 b, 1220 a,and 1220 b, respectively. Although the second coding units 1210 a and1210 b are determined by splitting the first coding unit 1200 in avertical direction differently from the second coding units 1220 a and1220 b which are determined by splitting the first coding unit 1200 in ahorizontal direction, the third coding units 1216 a, 1216 b, 1216 c, and1216 d, and 1226 a, 1226 b, 1226 c, and 1226 d split therefromeventually show same-shaped coding units split from the first codingunit 1200. As such, by recursively splitting a coding unit in differentmanners based on the split shape mode information, the image decodingapparatus 100 may process a plurality of coding units in differentorders even when the coding units are eventually determined to be thesame shape.

FIG. 13 illustrates a process of determining a depth of a coding unit asa shape and size of the coding unit change, when the coding unit isrecursively split such that a plurality of coding units are determined,according to an embodiment.

According to an embodiment, the image decoding apparatus 100 maydetermine the depth of the coding unit, based on a predeterminedcriterion. For example, the predetermined criterion may be the length ofa long side of the coding unit. When the length of a long side of acoding unit before being split is 2n times (n>0) the length of a longside of a split current coding unit, the image decoding apparatus 100may determine that a depth of the current coding unit is increased froma depth of the coding unit before being split, by n. In the followingdescription, a coding unit having an increased depth is expressed as acoding unit of a deeper depth.

Referring to FIG. 13 , according to an embodiment, the image decodingapparatus 100 may determine a second coding unit 1302 and a third codingunit 1304 of deeper depths by splitting a square first coding unit 1300based on block shape information indicating a square shape (for example,the block shape information may be expressed as ‘0: SQUARE’). Assumingthat the size of the square first coding unit 1300 is 2N×2N, the secondcoding unit 1302 determined by dividing a width and height of the firstcoding unit 1300 to ½ may have a size of N×N. Furthermore, the thirdcoding unit 1304 determined by dividing a width and height of the secondcoding unit 1302 to ½ may have a size of N/2×N/2. In this case, a widthand height of the third coding unit 1304 are ¼ times those of the firstcoding unit 1300. When a depth of the first coding unit 1300 is D, adepth of the second coding unit 1302, the width and height of which are½ times those of the first coding unit 1300, may be D+1, and a depth ofthe third coding unit 1304, the width and height of which are ¼ timesthose of the first coding unit 1300, may be D+2.

According to an embodiment, the image decoding apparatus 100 maydetermine a second coding unit 1312 or 1322 and a third coding unit 1314or 1324 of deeper depths by splitting a non-square first coding unit1310 or 1320 based on block shape information indicating a non-squareshape (for example, the block shape information may be expressed as ‘1:NS_VER’ indicating a non-square shape, a height of which is longer thana width, or as ‘2: NS_HOR’ indicating a non-square shape, a width ofwhich is longer than a height).

The image decoding apparatus 100 may determine a second coding unit1302, 1312, or 1322 by dividing at least one of a width and height ofthe first coding unit 1310 having a size of N×2N. That is, the imagedecoding apparatus 100 may determine the second coding unit 1302 havinga size of N×N or the second coding unit 1322 having a size of N×N/2 bysplitting the first coding unit 1310 in a horizontal direction, or maydetermine the second coding unit 1312 having a size of N/2×N bysplitting the first coding unit 1310 in horizontal and verticaldirections.

According to an embodiment, the image decoding apparatus 100 maydetermine the second coding unit 1302, 1312, or 1322 by dividing atleast one of a width and height of the first coding unit 1320 having asize of 2N×N. That is, the image decoding apparatus 100 may determinethe second coding unit 1302 having a size of N×N or the second codingunit 1312 having a size of N/2×N by splitting the first coding unit 1320in a vertical direction, or may determine the second coding unit 1322having a size of N×N/2 by splitting the first coding unit 1320 inhorizontal and vertical directions.

According to an embodiment, the image decoding apparatus 100 maydetermine a third coding unit 1304, 1314, or 1324 by dividing at leastone of a width and height of the second coding unit 1302 having a sizeof N×N. That is, the image decoding apparatus 100 may determine thethird coding unit 1304 having a size of N/2×N/2, the third coding unit1314 having a size of N/4×N/2, or the third coding unit 1324 having asize of N/2×N/4 by splitting the second coding unit 1302 in vertical andhorizontal directions.

According to an embodiment, the image decoding apparatus 100 maydetermine the third coding unit 1304, 1314, or 1324 by dividing at leastone of a width and height of the second coding unit 1312 having a sizeof N/2×N. That is, the image decoding apparatus 100 may determine thethird coding unit 1304 having a size of N/2×N/2 or the third coding unit1324 having a size of N/2×N/4 by splitting the second coding unit 1312in a horizontal direction, or may determine the third coding unit 1314having a size of N/4×N/2 by splitting the second coding unit 1312 invertical and horizontal directions.

According to an embodiment, the image decoding apparatus 100 maydetermine the third coding unit 1304, 1314, or 1324 by dividing at leastone of a width and height of the second coding unit 1322 having a sizeof N×N/2. That is, the image decoding apparatus 100 may determine thethird coding unit 1304 having a size of N/2×N/2 or the third coding unit1314 having a size of N/4×N/2 by splitting the second coding unit 1322in a vertical direction, or may determine the third coding unit 1324having a size of N/2×N/4 by splitting the second coding unit 1322 invertical and horizontal directions.

According to an embodiment, the image decoding apparatus 100 may splitthe square coding unit 1300, 1302, or 1304 in a horizontal or verticaldirection. For example, the image decoding apparatus 100 may determinethe first coding unit 1310 having a size of N×2N by splitting the firstcoding unit 1300 having a size of 2N×2N, in a vertical direction, or maydetermine the first coding unit 1320 having a size of 2N×N by splittingthe first coding unit 1300, in a horizontal direction. According to anembodiment, when a depth is determined based on the length of thelongest side of a coding unit, a depth of a coding unit determined bysplitting the first coding unit 1300 having a size of 2N×2N in ahorizontal or vertical direction may be the same as the depth of thefirst coding unit 1300.

According to an embodiment, a width and height of the third coding unit1314 or 1324 may be ¼ times those of the first coding unit 1310 or 1320.When a depth of the first coding unit 1310 or 1320 is D, a depth of thesecond coding unit 1312 or 1322, the width and height of which are ½times those of the first coding unit 1310 or 1320, may be D+1, and adepth of the third coding unit 1314 or 1324, the width and height ofwhich are ¼ times those of the first coding unit 1310 or 1320, may beD+2.

FIG. 14 illustrates depths that are determinable based on shapes andsizes of coding units, and part indexes (PIDs) that are fordistinguishing the coding units, according to an embodiment.

According to an embodiment, the image decoding apparatus 100 maydetermine various-shaped second coding units by splitting a square firstcoding unit 1400. Referring to FIG. 14 , the image decoding apparatus100 may determine second coding units 1402 a and 1402 b, 1404 a and 1404b, and 1406 a, 1406 b, 1406 c, and 1406 d by splitting the first codingunit 1400 in at least one of vertical and horizontal directions based onsplit shape mode information. That is, the image decoding apparatus 100may determine the second coding units 1402 a and 1402 b, 1404 a and 1404b, and 1406 a, 1406 b, 1406 c, and 1406 d, based on the split shape modeinformation of the first coding unit 1400.

According to an embodiment, a depth of the second coding units 1402 aand 1402 b, 1404 a and 1404 b, and 1406 a, 1406 b, 1406 c, and 1406 d,which are determined based on the split shape mode information of thesquare first coding unit 1400, may be determined based on the length ofa long side thereof. For example, because the length of a side of thesquare first coding unit 1400 equals the length of a long side of thenon-square second coding units 1402 a and 1402 b, and 1404 a and 1404 b,the first coding unit 1400 and the non-square second coding units 1402 aand 1402 b, and 1404 a and 1404 b may have the same depth, e.g., D.However, when the image decoding apparatus 100 splits the first codingunit 1400 into the four square second coding units 1406 a, 1406 b, 1406c, and 1406 d based on the split shape mode information, because thelength of a side of the square second coding units 1406 a, 1406 b, 1406c, and 1406 d is ½ times the length of a side of the first coding unit1400, a depth of the second coding units 1406 a, 1406 b, 1406 c, and1406 d may be D+1 which is deeper than the depth D of the first codingunit 1400 by 1.

According to an embodiment, the image decoding apparatus 100 maydetermine a plurality of second coding units 1412 a and 1412 b, and 1414a, 1414 b, and 1414 c by splitting a first coding unit 1410, a height ofwhich is longer than a width, in a horizontal direction based on thesplit shape mode information. According to an embodiment, the imagedecoding apparatus 100 may determine a plurality of second coding units1422 a and 1422 b, and 1424 a, 1424 b, and 1424 c by splitting a firstcoding unit 1420, a width of which is longer than a height, in avertical direction based on the split shape mode information.

According to an embodiment, a depth of the second coding units 1412 aand 1412 b, 1414 a, 1414 b, and 1414 c, 1422 a, 1422 b, 1424 a, 1424 b,and 1424 c, which are determined based on the split shape modeinformation of the non-square first coding unit 1410 or 1420, may bedetermined based on the length of a long side thereof. For example,because the length of a side of the square second coding units 1412 aand 1412 b is ½ times the length of a long side of the first coding unit1410 having a non-square shape, a height of which is longer than awidth, a depth of the square second coding units 1412 a and 1412 b isD+1 which is deeper than the depth D of the non-square first coding unit1410 by 1.

Furthermore, the image decoding apparatus 100 may split the non-squarefirst coding unit 1410 into an odd number of second coding units 1414 a,1414 b, and 1414 c based on the split shape mode information. The oddnumber of second coding units 1414 a, 1414 b, and 1414 c may include thenon-square second coding units 1414 a and 1414 c and the square secondcoding unit 1414 b. In this case, because the length of a long side ofthe non-square second coding units 1414 a and 1414 c and the length of aside of the square second coding unit 1414 b are ½ times the length of along side of the first coding unit 1410, a depth of the second codingunits 1414 a, 1414 b, and 1414 c may be D+1 which is deeper than thedepth D of the non-square first coding unit 1410 by 1. The imagedecoding apparatus 100 may determine depths of coding units split fromthe first coding unit 1420 having a non-square shape, a width of whichis longer than a height, by using the above-described method ofdetermining depths of coding units split from the first coding unit1410.

According to an embodiment, the image decoding apparatus 100 maydetermine PIDs for identifying split coding units, based on a size ratiobetween the coding units when an odd number of split coding units do nothave equal sizes. Referring to FIG. 14 , a coding unit 1414 b of acenter location among an odd number of split coding units 1414 a, 1414b, and 1414 c may have a width equal to that of the other coding units1414 a and 1414 c and a height which is two times that of the othercoding units 1414 a and 1414 c. That is, in this case, the coding unit1414 b at the center location may include two of the other coding unit1414 a or 1414 c. Therefore, when a PID of the coding unit 1414 b at thecenter location is 1 based on a scan order, a PID of the coding unit1414 c located next to the coding unit 1414 b may be increased by 2 andthus may be 3. That is, discontinuity in PID values may be present.According to an embodiment, the image decoding apparatus 100 maydetermine whether an odd number of split coding units do not have equalsizes, based on whether discontinuity is present in PIDs for identifyingthe split coding units.

According to an embodiment, the image decoding apparatus 100 maydetermine whether to use a specific splitting method, based on PIDvalues for identifying a plurality of coding units determined bysplitting a current coding unit. Referring to FIG. 14 , the imagedecoding apparatus 100 may determine an even number of coding units 1412a and 1412 b or an odd number of coding units 1414 a, 1414 b, and 1414 cby splitting the first coding unit 1410 having a rectangular shape, aheight of which is longer than a width. The image decoding apparatus 100may use PIDs to identify respective coding units. According to anembodiment, the PID may be obtained from a sample of a predeterminedlocation of each coding unit (e.g., a top left sample).

According to an embodiment, the image decoding apparatus 100 maydetermine a coding unit at a predetermined location from among the splitcoding units, by using the PIDs for distinguishing the coding units.According to an embodiment, when the split shape mode information of thefirst coding unit 1410 having a rectangular shape, a height of which islonger than a width, indicates to split a coding unit into three codingunits, the image decoding apparatus 100 may split the first coding unit1410 into three coding units 1414 a, 1414 b, and 1414 c. The imagedecoding apparatus 100 may assign a PID to each of the three codingunits 1414 a, 1414 b, and 1414 c. The image decoding apparatus 100 maycompare PIDs of an odd number of split coding units to determine acoding unit at a center location from among the coding units. The imagedecoding apparatus 100 may determine the coding unit 1414 b having a PIDcorresponding to a middle value among the PIDs of the coding units, asthe coding unit at the center location from among the coding unitsdetermined by splitting the first coding unit 1410. According to anembodiment, the image decoding apparatus 100 may determine PIDs fordistinguishing split coding units, based on a size ratio between thecoding units when the split coding units do not have equal sizes.Referring to FIG. 14 , the coding unit 1414 b generated by splitting thefirst coding unit 1410 may have a width equal to that of the othercoding units 1414 a and 1414 c and a height which is two times that ofthe other coding units 1414 a and 1414 c. In this case, when the PID ofthe coding unit 1414 b at the center location is 1, the PID of thecoding unit 1414 c located next to the coding unit 1414 b may beincreased by 2 and thus may be 3. When the PID is not uniformlyincreased as described above, the image decoding apparatus 100 maydetermine that a coding unit is split into a plurality of coding unitsincluding a coding unit having a size different from that of the othercoding units. According to an embodiment, when the split shape modeinformation indicates to split a coding unit into an odd number ofcoding units, the image decoding apparatus 100 may split a currentcoding unit in such a manner that a coding unit of a predeterminedlocation among an odd number of coding units (e.g., a coding unit of acentre location) has a size different from that of the other codingunits. In this case, the image decoding apparatus 100 may determine thecoding unit of the centre location, which has a different size, by usingPIDs of the coding units. However, the PIDs and the size or location ofthe coding unit of the predetermined location are not limited to theabove-described examples, and various PIDs and various locations andsizes of coding units may be used.

According to an embodiment, the image decoding apparatus 100 may use apredetermined data unit where a coding unit starts to be recursivelysplit.

FIG. 15 illustrates that a plurality of coding units are determinedbased on a plurality of predetermined data units included in a picture,according to an embodiment.

According to an embodiment, a predetermined data unit may be defined asa data unit where a coding unit starts to be recursively split by usingsplit shape mode information. That is, the predetermined data unit maycorrespond to a coding unit of an uppermost depth, which is used todetermine a plurality of coding units split from a current picture. Inthe following descriptions, for convenience of explanation, thepredetermined data unit is referred to as a reference data unit.

According to an embodiment, the reference data unit may have apredetermined size and a predetermined size shape. According to anembodiment, a reference coding unit may include M×N samples. Herein, Mand N may be equal to each other, and may be integers expressed aspowers of 2. That is, the reference data unit may have a square ornon-square shape, and may be split into an integer number of codingunits.

According to an embodiment, the image decoding apparatus 100 may splitthe current picture into a plurality of reference data units. Accordingto an embodiment, the image decoding apparatus 100 may split theplurality of reference data units, which are split from the currentpicture, by using split shape mode information about each reference dataunit. The operation of splitting the reference data unit may correspondto a splitting operation using a quadtree structure.

According to an embodiment, the image decoding apparatus 100 maypreviously determine the minimum size allowed for the reference dataunits included in the current picture. Accordingly, the image decodingapparatus 100 may determine various reference data units having sizesequal to or greater than the minimum size, and may determine one or morecoding units by using the split shape mode information with reference tothe determined reference data unit.

Referring to FIG. 15 , the image decoding apparatus 100 may use a squarereference coding unit 1500 or a non-square reference coding unit 1502.According to an embodiment, the shape and size of reference coding unitsmay be determined based on various data units capable of including oneor more reference coding units (e.g., sequences, pictures, slices, slicesegments, coding tree units, or the like).

According to an embodiment, the receiver (not shown) of the imagedecoding apparatus 100 may obtain, from a bitstream, at least one ofreference coding unit shape information and reference coding unit sizeinformation with respect to each of the various data units. An operationof splitting the square reference coding unit 1500 into one or morecoding units has been described above in relation to the operation ofsplitting the current coding unit 300 of FIG. 3 , and an operation ofsplitting the non-square reference coding unit 1502 into one or morecoding units has been described above in relation to the operation ofsplitting the current coding unit 400 or 450 of FIG. 4 . Thus, detaileddescriptions thereof will not be provided herein.

According to an embodiment, the image decoding apparatus 100 may use aPID for identifying the size and shape of reference coding units, todetermine the size and shape of reference coding units according to somedata units previously determined based on a predetermined condition.That is, the receiver (not shown) may obtain, from the bitstream, onlythe PID for identifying the size and shape of reference coding unitswith respect to each slice, slice segment, or coding tree unit which isa data unit satisfying a predetermined condition (e.g., a data unithaving a size equal to or smaller than a slice) among the various dataunits (e.g., sequences, pictures, slices, slice segments, coding treeunits, or the like). The image decoding apparatus 100 may determine thesize and shape of reference data units with respect to each data unit,which satisfies the predetermined condition, by using the PID. When thereference coding unit shape information and the reference coding unitsize information are obtained and used from the bitstream according toeach data unit having a relatively small size, efficiency of using thebitstream may not be high, and therefore, only the PID may be obtainedand used instead of directly obtaining the reference coding unit shapeinformation and the reference coding unit size information. In thiscase, at least one of the size and shape of reference coding unitscorresponding to the PID for identifying the size and shape of referencecoding units may be previously determined. That is, the image decodingapparatus 100 may determine at least one of the size and shape ofreference coding units included in a data unit serving as a unit forobtaining the PID, by selecting the previously determined at least oneof the size and shape of reference coding units based on the PID.

According to an embodiment, the image decoding apparatus 100 may use oneor more reference coding units included in a coding tree unit. That is,a coding tree unit split from a picture may include one or morereference coding units, and coding units may be determined byrecursively splitting each reference coding unit. According to anembodiment, at least one of a width and height of the coding tree unitmay be integer times at least one of the width and height of thereference coding units. According to an embodiment, the size ofreference coding units may be obtained by splitting the coding tree unitn times based on a quadtree structure. That is, the image decodingapparatus 100 may determine the reference coding units by splitting thecoding tree unit n times based on a quadtree structure, and may splitthe reference coding unit based on at least one of the block shapeinformation and the split shape mode information according to variousembodiments.

FIG. 16 illustrates a processing block serving as a unit for determininga determination order of reference coding units included in a picture1600, according to an embodiment.

According to an embodiment, the image decoding apparatus 100 maydetermine one or more processing blocks split from a picture. Theprocessing block is a data unit including one or more reference codingunits split from a picture, and the one or more reference coding unitsincluded in the processing block may be determined according to aspecific order. That is, a determination order of one or more referencecoding units determined in each processing block may correspond to oneof various types of orders for determining reference coding units, andmay vary depending on the processing block. The determination order ofreference coding units, which is determined with respect to eachprocessing block, may be one of various orders, e.g., raster scan order,Z-scan, N-scan, up-right diagonal scan, horizontal scan, and verticalscan, but is not limited to the above-mentioned scan orders.

According to an embodiment, the image decoding apparatus 100 may obtainprocessing block size information and may determine the size of one ormore processing blocks included in the picture. The image decodingapparatus 100 may obtain the processing block size information from abitstream and may determine the size of one or more processing blocksincluded in the picture. The size of processing blocks may be apredetermined size of data units, which is indicated by the processingblock size information.

According to an embodiment, the receiver (not shown) of the imagedecoding apparatus 100 may obtain the processing block size informationfrom the bitstream according to each specific data unit. For example,the processing block size information may be obtained from the bitstreamin a data unit such as an image, sequence, picture, slice, or slicesegment. That is, the receiver (not shown) may obtain the processingblock size information from the bitstream according to each of thevarious data units, and the image decoding apparatus 100 may determinethe size of one or more processing blocks, which are split from thepicture, by using the obtained processing block size information. Thesize of the processing blocks may be integer times that of the referencecoding units.

According to an embodiment, the image decoding apparatus 100 maydetermine the size of processing blocks 1602 and 1612 included in thepicture 1600. For example, the image decoding apparatus 100 maydetermine the size of processing blocks based on the processing blocksize information obtained from the bitstream. Referring to FIG. 16 ,according to an embodiment, the image decoding apparatus 100 maydetermine a width of the processing blocks 1602 and 1612 to be fourtimes the width of the reference coding units, and may determine aheight of the processing blocks 1602 and 1612 to be four times theheight of the reference coding units. The image decoding apparatus 100may determine a determination order of one or more reference codingunits in one or more processing blocks.

According to an embodiment, the image decoding apparatus 100 maydetermine the processing blocks 1602 and 1612, which are included in thepicture 1600, based on the size of processing blocks, and may determinea determination order of one or more reference coding units in theprocessing blocks 1602 and 1612. According to an embodiment,determination of reference coding units may include determination of thesize of the reference coding units.

According to an embodiment, the image decoding apparatus 100 may obtain,from the bitstream, determination order information of one or morereference coding units included in one or more processing blocks, andmay determine a determination order with respect to one or morereference coding units based on the obtained determination orderinformation. The determination order information may be defined as anorder or direction for determining the reference coding units in theprocessing block. That is, the determination order of reference codingunits may be independently determined with respect to each processingblock.

According to an embodiment, the image decoding apparatus 100 may obtain,from the bitstream, the determination order information of referencecoding units according to each specific data unit. For example, thereceiver (not shown) may obtain the determination order information ofreference coding units from the bitstream according to each data unitsuch as an image, sequence, picture, slice, slice segment, or processingblock. Because the determination order information of reference codingunits indicates an order for determining reference coding units in aprocessing block, the determination order information may be obtainedwith respect to each specific data unit including an integer number ofprocessing blocks.

According to an embodiment, the image decoding apparatus 100 maydetermine one or more reference coding units based on the determineddetermination order.

According to an embodiment, the receiver (not shown) may obtain thedetermination order information of reference coding units from thebitstream as information related to the processing blocks 1602 and 1612,and the image decoding apparatus 100 may determine a determination orderof one or more reference coding units included in the processing blocks1602 and 1612 and determine one or more reference coding units, whichare included in the picture 1600, based on the determination order.Referring to FIG. 16 , the image decoding apparatus 100 may determinedetermination orders 1604 and 1614 of one or more reference coding unitsin the processing blocks 1602 and 1612, respectively. For example, whenthe determination order information of reference coding units isobtained with respect to each processing block, different types of thedetermination order information of reference coding units may beobtained for the processing blocks 1602 and 1612. When the determinationorder 1604 of reference coding units in the processing block 1602 is araster scan order, reference coding units included in the processingblock 1602 may be determined according to a raster scan order. On thecontrary, when the determination order 1614 of reference coding units inthe other processing block 1612 is a backward raster scan order,reference coding units included in the processing block 1612 may bedetermined according to the backward raster scan order.

According to an embodiment, the image decoding apparatus 100 may decodethe determined one or more reference coding units. The image decodingapparatus 100 may decode an image, based on the reference coding unitsdetermined as described above. A method of decoding the reference codingunits may include various image decoding methods.

According to an embodiment, the image decoding apparatus 100 may obtainblock shape information indicating the shape of a current coding unit orsplit shape mode information indicating a splitting method of thecurrent coding unit, from the bitstream, and may use the obtainedinformation. The split shape mode information may be included in thebitstream related to various data units. For example, the image decodingapparatus 100 may use the block shape information or the split shapemode information included in a sequence parameter set, a pictureparameter set, a video parameter set, a slice header, or a slice segmentheader. Furthermore, the image decoding apparatus 100 may obtain, fromthe bitstream, a syntax element corresponding to the block shapeinformation or the split shape mode information according to each codingtree unit, each reference coding unit, or each processing block, and mayuse the obtained syntax element.

Hereinafter, a method of determining a splitting rule according to anembodiment of the present disclosure will be described in detail.

The image decoding apparatus 100 may determine a splitting rule forimages. The splitting rule may be previously determined between theimage decoding apparatus 100 and the image encoding apparatus 150. Theimage decoding apparatus 100 may determine a splitting rule of imagesbased on information obtained from a bitstream. The image decodingapparatus 100 may determine a splitting rule based on informationobtained from at least one of a sequence parameter set, a pictureparameter set, a video parameter set, a slice header, or a slice segmentheader. The image decoding apparatus 100 may determine differentsplitting rules according to a frame, a slice, a temporal layer, acoding tree unit or a coding unit.

The image decoding apparatus 100 may determine a splitting rule based ona block shape of a coding unit. The block shape may include a size,shape, a ratio between a width and a height, and a direction of a codingunit. The image encoding apparatus 150 and the image decoding apparatus100 may previously determine to determine a splitting rule based on ablock shape of a coding unit. However, the disclosure is not limitedthereto. The image decoding apparatus 100 may determine a splitting rulebased on information obtained from a bitstream received from the imageencoding apparatus 150.

A shape of a coding unit may include a square and a non-square. When awidth and a height of a coding unit have an equal length, the imagedecoding apparatus 100 may determine a shape of the coding unit to be asquare. In addition, when lengths of a width and a height of a codingunit are not equal, the image decoding apparatus 100 may determine ashape of the coding unit to be a non-square.

A size of a coding unit may include various sizes such as 4×4, 8×4, 4×8,8×8, 16×4, 16×8, . . . , 256×256. The size of the coding unit may beclassified according to the length of the long side, the length or thewidth of the short side of the coding unit. The image decoding apparatus100 may apply the same splitting rule to coding units classified intothe same group. For example, the image decoding apparatus 100 mayclassify coding units having the same long side length into the samesize. Also, the image decoding apparatus 100 may apply the samesplitting rule to coding units having the same long side length.

The ratio of the width and height of a coding unit may include 1:2, 2:1,1:4, 4:1, 1:8, 8:1, 1:16, or 16:1. In addition, a direction of a codingunit may include a horizontal direction and a vertical direction. Ahorizontal direction may represent a case where a length of a width of acoding unit is longer than a length of a height thereof. A verticaldirection may represent a case where a length of a width of a codingunit is shorter than a length of a height thereof.

The image decoding apparatus 100 may adaptively determine a splittingrule based on a size of a coding unit. The image decoding apparatus 100may differently determine an allowable splitting shape mode based on asize of a coding unit. For example, the image decoding apparatus 100 maydetermine whether splitting is allowed based on a size of a coding unit.The image decoding apparatus 100 may determine a split directionaccording to a size of a coding unit. The image decoding apparatus 100may determine an allowable split type according to a size of a codingunit.

The determination of the splitting rule based on the size of the codingunit may be a splitting rule that is previously determined between theimage encoding apparatus 150 and the image decoding apparatus 100 Also,the image decoding apparatus 100 may determine a splitting rule based oninformation obtained from a bitstream.

The image decoding apparatus 100 may adaptively determine a splittingrule based on a position of a coding unit. The image decoding apparatus100 may adaptively determine a splitting rule based on a position of acoding unit in an image.

In addition, the image decoding apparatus 100 may determine a splittingrule such that coding units generated by different splitting paths donot have the same block shape. However, the disclosure is not limitedthereto, and coding units generated by different splitting paths mayalso have the same block shape. Coding units generated by differentsplitting paths may have different decoding processing orders. As thedecoding processing orders have been described with reference to FIG. 12, detailed description thereof will be omitted.

Hereinafter, with reference to FIGS. 17 through 29 , an imageencoding/decoding method and apparatus for determining a filteringreference sample to be filtered and a weight of a filter and adaptivelyperforming intra prediction based on the filtering reference sample andthe weight of the filter will be described in detail.

FIG. 17 is a diagram for describing intra prediction modes according toan embodiment.

Referring to FIG. 17 , intra prediction modes according to an embodimentmay include a planar mode (mode No. 0), a DC mode (mode No. 1). Inaddition, the intra prediction modes may include angular modes having aprediction direction (modes Nos. 2 to 66). The angular modes may includea diagonal mode (mode No. 2 or 66), a horizontal mode (mode No. 18), anda vertical mode (mode No. 50).

While intra prediction modes according to an embodiment are describedwith reference to FIG. 17 , the disclosure is not limited thereto,various forms of intra prediction modes may be provided by adding a newintra prediction mode or removing a previous intra prediction mode, andthe mode number of each intra prediction mode may vary as is obvious toone of ordinary skill in the art.

FIG. 18 is a diagram for describing a method, performed by an imagedecoding apparatus, of generating a sample reconstructed using originalreference samples, according to an embodiment of the present disclosure.

Referring to FIG. 18 , to perform intra prediction on a current block1800, the image decoding apparatus 100 may generate samples 1820reconstructed using original reference samples 1810.

For example, the image decoding apparatus 100 may generate areconstructed reference sample 1835 corresponding to a position of anoriginal reference sample, by using an original reference sample 1830.The image decoding apparatus 100 may generate a reconstructed referencesample 1845 corresponding to a position of an original reference sample1840 by using the original reference sample 1830 and the originalreference sample 1840. In a similar manner to the above, the imagedecoding apparatus 100 may generate a reconstructed reference sample1855 at a position corresponding to an original reference sample 1850.Here, a reconstructed reference sample 1855 may be generated using theoriginal reference sample 1850 and original reference samples at theleft side of the original reference sample 1850, from among the originalreference samples 1810. That is, the image decoding apparatus 100 maygenerate a reconstructed reference sample a′_(n) based on Equation 1 asbelow.

a′ _(n)=Σ_(i=0) ^(n) w _(i) a _(i)(Σ_(i=0) ^(n) w _(i)=1)  [Equation 1]

Here, a′_(n) may denote a reference sample that is away by n from areconstructed reference sample that is at the leftmost side from amongreconstructed reference samples of an upper adjacent line of a currentblock. a_(i) may denote a reference sample that is away by i from anoriginal reference sample that is at the leftmost side from among theoriginal reference samples of the upper adjacent line. W_(i) may denotea filter weight applied to a sample a_(i).

A reconstructed reference sample may be generated by performingfiltering on at least one original reference sample as described above.

Referring to FIG. 18 , while an example of generating a reconstructedreference sample is described above, the disclosure is not limitedthereto, and a reconstructed reference sample may be generated byperforming filtering by using the same weight and a filter of the samenumber of filter tabs. For example, a reconstructed reference sample maybe generated by performing filtering by using a [1, 4] filter.

Alternatively, the image decoding apparatus 100 may adaptively determinea weight of a filter and a number of filter tabs according to a positionof an original reference sample or an intra prediction mode of a currentblock, and a reconstructed reference sample may be generated byperforming filtering based on the weight of the filter and the number offilter tabs. Alternatively, a reconstructed reference sample may begenerated by adaptively determining a weight of a filter and a number offilter tabs according to a size of a current block and performingfiltering based on the weight of the filter and the number of filtertabs.

FIGS. 19A and 19B are diagrams for describing a method, performed by theimage decoding apparatus 100, of generating a sample reconstructed usingoriginal reference samples in a prediction direction of an intraprediction mode of a current block, according to an embodiment of thepresent disclosure.

Referring to FIGS. 19A and 19B, when a direction of an intra predictionmode of a current block is a prediction direction 1905, the imagedecoding apparatus 100 may determine original reference samples 1925 ofan upper adjacent line on which filtering is to be performed to generatereconstructed reference samples 1920 of the upper adjacent line, basedon an x-axis direction 1930 of the prediction direction 1905. Forexample, when a direction of an intra prediction mode of a current blockis the prediction direction 1905, the image decoding apparatus 100 maygenerate a reconstructed sample a′_(j) according to Equation 2 as below.

a′ _(j)=Σ_(i=0) ^(j) w _(i) a _(i)(Σ_(i=0) ^(j) w _(i)=1)  [Equation 2]

Here, a′_(j) may denote a reference sample that is away by j from areconstructed reference sample that is at the leftmost side from amongthe reconstructed reference samples of an upper adjacent line of acurrent block. a_(i) may denote an original reference sample that isaway by i from an original reference sample that is at the leftmost sidefrom among the original reference samples of the upper adjacent line ofthe current block. W_(i) may denote a filter weight applied to a samplea_(i).

When a direction of an intra prediction mode of the current block is aprediction direction 1910, the image decoding apparatus 100 maydetermine the original reference samples 1925 of the upper adjacentline, on which filtering is to be performed to generate thereconstructed reference samples 1920 of the upper adjacent line, basedon an x-axis direction 1935 of the prediction direction 1910. When adirection of the intra prediction mode of the current block is theprediction direction 1910, the image decoding apparatus 100 may generatea reconstructed sample a′_(j) according to Equation 3 as below.

a′ _(j)Σ_(i=N) ^(j) w _(i)(Σ_(i=N) ^(j) w _(i)=1)  [Equation 3]

Here, a′_(j) may denote an original reference sample that is away by jfrom a reconstructed reference sample that is at the leftmost side fromamong the reconstructed reference samples of the upper adjacent line ofthe current block. a_(i) may denote an original reference sample that isaway by i from an original reference sample that is at the leftmost sidefrom among the original reference samples of the upper adjacent line ofthe current block, and N may denote a distance of a sample that isfarthest from a reconstructed reference sample located at the leftmostside.

When a direction of the intra prediction mode of the current block is aprediction direction 1915, the image decoding apparatus 100 maydetermine the original reference samples 1925 of the upper adjacentline, on which filtering is to be performed to generate thereconstructed reference samples 1920 of the upper adjacent line, basedon the x-axis direction 1935 of the prediction direction 1915. When adirection of the intra prediction mode of the current block is theprediction direction 1915, the image decoding apparatus 100 may generatea reconstructed sample a′_(j) according to Equation 4 as below.

a′ _(j)=Σ_(i=N) ^(j) w _(i) a _(i)(Σ_(i=N) ^(j) w _(i)=1)  [Equation 4]

Here, a′_(j) may denote an original reference sample that is away by jfrom a reconstructed reference sample that is at the leftmost side fromamong the reconstructed reference samples of the upper adjacent line theof a current block. a_(i) may denote an original reference sample thatis away by i from an original reference sample that is at the leftmostside from among the original reference samples of the upper adjacentline of the current block, and N may denote a distance of a sample thatis farthest from a reconstructed reference sample located at theleftmost side.

The image decoding apparatus 100 may generate reconstructed referencesamples of a left adjacent line in a similar manner to generating of thereconstructed reference samples of the upper adjacent line.

When a direction of the intra prediction mode of the current block is aprediction direction 1905, the image decoding apparatus 100 maydetermine original reference samples 1945 of the left adjacent line, onwhich filtering is to be performed to generate reconstructed referencesamples 1940 of the left adjacent line, based on an y-axis direction1950 of the prediction direction 1905.

When a direction of the intra prediction mode of the current block isthe prediction direction 1910, the image decoding apparatus 100 maydetermine the original reference samples 1945 of the left adjacent line,on which filtering is to be performed to generate the reconstructedreference samples 1940 of the left adjacent line, based on the y-axisdirection 1950 of the prediction direction 1910.

When a direction of the intra prediction mode of the current block isthe prediction direction 1915, the image decoding apparatus 100 maydetermine the original reference samples 1945 of the left adjacent line,on which filtering is to be performed to generate the reconstructedreference samples 1940 of the left adjacent line, based on a y-axisdirection 1955 of the prediction direction 1915.

FIG. 20 is a diagram for describing a method, performed by an imagedecoding apparatus, of generating a sample reconstructed using originalreference samples, according to an embodiment of the present disclosure.

Referring to FIG. 20 , to perform intra prediction on a current block2000, the image decoding apparatus 100 may generate samples a′₀, . . . ,a′_(N) reconstructed using original reference samples a₀, . . . , a_(N).

When a prediction direction of an intra prediction mode of the currentblock 2000 is from the left to the right (that is, when the predictiondirection is a direction located in a first quadrant of FIG. 19A), theimage decoding apparatus 100 may generate a reconstructed referencesample 2015 corresponding to a position of an original reference sample2010 by using the original reference sample 2010. The image decodingapparatus 100 may generate a reconstructed reference sample 2025corresponding to a position of an original reference sample 2020 byusing the original reference sample 2010 and the original referencesample 2020. In a similar manner to the above, the image decodingapparatus 100 may generate a reconstructed reference sample 2035 at aposition corresponding to an original reference sample 2030. Here, fromamong the original reference samples, the reconstructed reference sample2035 may be generated using the original reference sample 2030 and anoriginal reference sample 2031 immediately adjacent on the left thereof.That is, the image decoding apparatus 100 may generate a reconstructedreference sample a′_(j) based on Equation 5 as below.

a′ _(i)=¼(a _(j−1)+3a _(j))  [Equation 5]

Here, a′_(j) may denote a reconstructed reference sample that is away byj from a reconstructed reference sample that is at the leftmost sidefrom among the reconstructed reference samples of an upper adjacent lineof a current block, and a_(j) or a_(j−1) may denote an originalreference sample that is away by j or j−1 from an original referencesample that is at the leftmost side from among the original referencesamples of the upper adjacent line of the current block.

When a prediction direction of an intra prediction mode of the currentblock 2000 is from the right to the left (that is, when the predictiondirection is a direction located in a second quadrant or a thirdquadrant of FIG. 19A), the image decoding apparatus 100 may generate areference sample 2045 corresponding to a position of an originalreference sample 2040 by using the original reference sample 2040. Theimage decoding apparatus 100 may generate a reconstructed referencesample 2055 corresponding to a position of an original reference sample2050 by using the original reference sample 2040 and the originalreference sample 2050. In a similar manner to the above, the imagedecoding apparatus 100 may generate the reconstructed reference sample2035 at a position corresponding to the original reference sample 2030.Here, from among the original reference samples, the reconstructedreference sample 2035 may be generated using the original referencesample 2030 and an original reference sample 2032 immediately adjacenton the right thereof. That is, the image decoding apparatus 100 maygenerate a reconstructed reference sample a′_(j) based on Equation 6 asbelow.

a′ _(j)=¼(3a _(j) +a _(j+1))  [Equation 6]

Here, a′_(j) may denote a reconstructed reference sample that is away byj from a reconstructed reference sample that is at the leftmost sidefrom among the reconstructed reference samples of the upper adjacentline of the current block, and a_(j) or a_(j+1) may denote an originalreference sample that is away by j or j+1 from an original referencesample that is at the leftmost side from among the original referencesamples of the upper adjacent line of the current block.

The image decoding apparatus 100 may generate reconstructed referencesamples of a left adjacent line in a similar manner to generating of thereconstructed reference samples of the upper adjacent line. When ay-axis direction of a prediction direction of an intra prediction modeof a current block is from bottom to top (that is, when the predictiondirection is a direction located in the first quadrant or the secondquadrant of FIG. 19A), the image decoding apparatus 100 may generatereconstructed reference samples of the left adjacent line in a similarmanner as when generating reconstructed reference samples of the upperadjacent line when the x-axis direction is from the right to the left.

The image decoding apparatus 100 may generate reconstructed referencesamples of the left adjacent line in a similar manner to generating ofthe reconstructed reference samples of the upper adjacent line. When ay-axis direction of the prediction direction of the intra predictionmode of the current block is from top to bottom (that is, when theprediction direction is a direction located in the third quadrant ofFIG. 19A), the image decoding apparatus 100 may generate referencesamples of the left adjacent line in a similar manner as when generatingthe reconstructed reference samples of the upper adjacent line when thex-axis direction is from the left to the right.

In a DC mode without directivity, a planar mode, or a vertical modehaving only a vertical direction component or a horizontal mode havingonly a horizontal direction component, the image decoding apparatus 100may generate a reconstructed reference sample a′_(j) of the left orupper adjacent line based on Equation 7 as below.

a′ _(j)=⅛(a _(j−1)+6a _(j) ++a _(j+1))  [Equation 7]

Here, a′_(j) may denote a reference sample that is away by j from areconstructed reference sample that is at the leftmost side from amongthe reconstructed reference samples of the upper adjacent line of thecurrent block, and a_(j−1), a_(j) or a_(j+1) may denote an originalreference sample that is away by j−1, j, or j+1 from an originalreference sample that is at the leftmost side from among the originalreference samples of the upper adjacent line of the current block.

The image decoding apparatus 100 may generate reconstructed samples byperforming filtering on an original reference sample, and may performintra prediction on a current block by using the reconstructed samples,thereby obtaining the effect of performing intra prediction by referringto more diverse reference samples than when performing intra predictionby using an original reference sample.

FIG. 21 is a diagram for describing a process, performed by an imagedecoding apparatus, of performing intra prediction on a current block byusing original reference samples and a reconstructed sample, accordingto an embodiment of the present disclosure.

The image decoding apparatus 100 may generate a prediction value of acurrent sample of a current block by using original reference samplesand reconstructed reference samples. For example, when an intraprediction mode of a current block 2100 is a vertical mode, the imagedecoding apparatus 100 may generate a prediction value of a currentsample 2110 by using an original reference sample 2121 that is above thecurrent sample 2110 and a reconstructed reference sample 2131. Forexample, the image decoding apparatus 100 may generate a predictionvalue p_(n) of the current sample based on Equation 9.

p _(n)=(wf(a _(n))+(1−w)f(a′ _(n)))  [Equation 9]

Here, p_(n) may denote a prediction value of the current sample, and anmay denote an original reference sample used in prediction of thecurrent sample, and a′_(n) may denote a reconstructed reference sampleused in prediction of the current sample, and w may denote a weight andhave a value between 0 and 1. Also, f may denote a prediction functionof a 4-tab filter or the like. Here, w may be a fixed value, but is notlimited thereto, and may vary for each sample of the current block, andmay be a value based on a distance from an original reference sample ora reconstructed reference sample.

While generation of a prediction value p_(n) of the current sample atonce by the image decoding apparatus 100 based on Equation 9 isdescribed above, the disclosure is not limited thereto, and the imagedecoding apparatus 100 may generate an initial prediction value byperforming intra prediction by using an original reference sample as inthe related art and generate a final prediction value p′_(n) of thecurrent sample by performing filtering by using the initial predictionvalue and a reconstructed reference sample.

Referring to FIG. 21 , while performing of intra prediction on a currentblock, by the image decoding apparatus 100, by using both originalreference samples and reconstructed reference samples, is described, thedisclosure is not limited thereto, and the image decoding apparatus 100may also perform intra prediction on a current block by using onlyoriginal reference samples. Alternatively, the image decoding apparatus100 may perform intra prediction on a current block by using onlyreconstructed reference samples.

Referring to FIG. 21 , while the above description is based on theassumption that a first intra prediction mode of a current block fordetermining an original reference sample used in intra prediction fromamong the original reference samples is identical to a second intraprediction mode for determining a reconstructed reference sample used inintra prediction from among reconstructed reference samples, thedisclosure is not limited thereto, and the second intra prediction modemay be determined separately from the first intra prediction mode. Forexample, an intra prediction mode for a reconstructed reference samplemay be determined for each block or for each intra prediction mode of acurrent block for determining an original reference sample used in intraprediction from among the original reference samples. Alternatively, asecond intra prediction mode may be determined in units of pictures.

FIG. 22 is a diagram for describing a process, performed by an imagedecoding apparatus, of performing weighted prediction by using anoriginal reference sample and reconstructed reference samples of a leftadjacent line and an upper adjacent line.

The image decoding apparatus 100 may determine an original referencesample to be used in intra prediction, from among the original referenceprediction samples based on an intra prediction mode of a current block2200, and determine a reconstructed sample of a left adjacent line ofthe current block 2200 and a reconstructed sample of an upper adjacentline of the current block 2200 that are to be used in intra prediction,regardless of an intra prediction mode of the current block 2200, andgenerate a prediction value with respect to a current sample 2210 byperforming weighted prediction on the current sample 2210 by using theoriginal reference sample and the reconstructed reference samples.

For example, referring to FIG. 22 , when an intra prediction mode of acurrent block is a vertical mode, the image decoding apparatus 100 maydetermine an original reference sample a_(j) located in a verticaldirection of the current sample 2210 from among original referencesamples 2220 of the upper adjacent line. The image decoding apparatus100 may determine a reconstructed reference sample a′_(j) located in thevertical direction of the current sample 2210. Also, the image decodingapparatus 100 may determine a reconstructed reference sample b′_(i)located in a horizontal direction of the current sample 2210. The imagedecoding apparatus 100 may generate a prediction value of the currentsample 2210 by using the determined original reference sample a_(j) andthe reconstructed reference samples a′_(j) and b′_(i). That is, theimage decoding apparatus 100 may generate a prediction value p_(ij) ofthe current sample based on Equation 10 below.

p _(ij) =w ₁ f(a′ _(j))+w ₂ f(b′ _(j))+w ₃ f(a _(j))  [Equation 10]

Here, p_(ij) may denote a prediction value of a current sample at aposition (i,j), a_(j) may denote the original reference sample used forprediction of the current sample, and a′_(j) may denote thereconstructed reference sample of the upper adjacent line used inprediction of the current sample, and b′_(i) may denote thereconstructed reference sample of the left adjacent line used inprediction of the current sample. Also, f may denote a predictionfunction of a 4-tab filter or the like. Here, w₁, w₂, and w₃ may bedetermined for each sample, and may be determined based on a distancebetween an original reference sample or a reconstructed reference sampleand a current sample.

While a process in which the image decoding apparatus 100 performsweighted prediction by using an original reference sample and areconstructed reference sample of a left adjacent line located in ahorizontal direction of a current sample and a reconstructed referencesample of an upper adjacent line located in a vertical direction of thecurrent sample is described with reference to FIG. 22 , the disclosureis not limited thereto, and the image decoding apparatus 100 maydetermine a prediction direction for selecting a reconstructed referencesample with respect to the current sample by considering a change in agradient of a reference sample. That is, the image decoding apparatus100 may determine a gradient direction of a reference sample that hasthe same change tendency as a gradient value of the reference sample tobe a prediction direction for selecting a reconstructed reference samplewith respect to a current sample.

FIG. 23 is a diagram for describing a process, performed by an imagedecoding apparatus, of performing weighted prediction by using aprediction value generated by performing intra prediction using anoriginal reference sample and reconstructed reference samples of a leftadjacent line and an upper adjacent line.

The image decoding apparatus 100 may generate an intermediate predictionvalue with respect to a current sample in a current block by performingintra prediction based on an original reference prediction sample basedon an intra prediction mode of a current block 2300. The image decodingapparatus 100 may determine at least one reference sample from amongreconstructed reference samples of the left adjacent line and at leastone reference sample from among reconstructed reference samples of theupper adjacent line, regardless of the intra prediction mode of thecurrent block 2300. The image decoding apparatus 100 may generate afinal prediction value with respect to a current sample in the currentblock 2300 by using an intermediate prediction value with respect to thecurrent sample in the current block 2300, the reconstructed referencesample of the left adjacent line, and the reconstructed reference sampleof the upper adjacent line.

The image decoding apparatus 100 may generate a final prediction valuep′ij of the current sample in the current block based on Equation 11.

p′ _(ij) =C _(ij) *[a′ _(i) p _(ij) a′ _(j) ]=[Ca′ _(i) [i,j]Cp _(ij)Ca′ _(j) [i,j]]*[a′ _(i) p _(ij) a′ _(′)]  [Equation 11]

Here, p_(ij) may denote an intermediate prediction value of a samplelocated at (i,j) in the current block generated by performing intraprediction by using original reference samples according to the intraprediction mode of the current block 2300. C_(ij) denotes a matrixincluding filter coefficients applied to p_(ij), a′_(i), and a′_(j).Ca′_(i)[i,j] may be a two-dimensional matrix including a filtercoefficient applied to a′_(i). Ca′_(j)[i,j] may be a two-dimensionalmatrix including a filter coefficient applied to a′_(j). Cp_(ij)[i,j]may denote a two-dimensional matrix including a filter coefficientapplied to p_(ij).

For example, referring to FIG. 23 , the image decoding apparatus maygenerate a final prediction value of the current sample in the currentblock 2300 by using Ca_(i)′[i,j](2310), Ca_(j)′ [i,j] (2320), andCp_(ij)[i,j](2330). However, while the filter coefficients disclosed inFIG. 23 are non-normalized coefficients, it will be obvious to one ofordinary skill in the art that, when the image decoding apparatus 100actually uses the filter coefficients disclosed in FIG. 23 , acoefficient generated by performing normalization (that is, performingan operation corresponding to an operation of division of filtercoefficients by 16) may be finally used.

That is, the image decoding apparatus 100 may generate a finalprediction value p′_(ij) of the current sample based on Equation 12 asbelow.

$\begin{matrix}{{p_{ij}^{\prime} = {\left\lbrack {{{Ca}\left\lbrack {i,j} \right\rbrack}{{Cp}\left\lbrack {i,j} \right\rbrack}{{Ca}\left\lbrack {i,j} \right\rbrack}} \right\rbrack*\left\lbrack {a_{i}^{\prime}p_{ij}a_{j}^{\prime}} \right\rbrack}},} & \left\lbrack {{Equation}12} \right\rbrack\end{matrix}$ ${where}\begin{matrix}{{{Ca}\left\lbrack {i,j} \right\rbrack} = \begin{matrix}\begin{matrix}\left\lbrack 1 \right. & 2 & \left. 1 \right\rbrack\end{matrix} \\\begin{matrix}\left\lbrack 5 \right. & 1 & \left. 1 \right\rbrack\end{matrix} \\\begin{matrix}\left\lbrack 5 \right. & 3 & \left. 1 \right\rbrack\end{matrix}\end{matrix}} & {{{Cp}\left\lbrack {i,j} \right\rbrack} = \begin{matrix}\begin{matrix}\left\lbrack 2 \right. & {9} & \left. 10 \right\rbrack\end{matrix} \\\begin{matrix}\left\lbrack 9 \right. & {6} & \left. 12 \right\rbrack\end{matrix} \\\begin{matrix}\left\lbrack 10 \right. & 12 & \left. 14 \right\rbrack\end{matrix}\end{matrix}}\end{matrix}$ ifiorj ≥ 3, coefficientfori = 3orj = 3isused.

Here, p_(ij) may denote an intermediate prediction value of a samplelocated at (i,j) in the current block generated by performing intraprediction by using original reference samples according to the intraprediction mode of the current block 2300. Ca [i,j] may be atwo-dimensional matrix including a filter coefficient applied to a′_(i)and a′_(j). Cp[i,j] may denote a two-dimensional metric including afilter coefficient applied to p_(ij).

Meanwhile, while a process in which the image decoding apparatus 100performes weighted prediction by using a prediction value generated byperforming intra prediction by using an original reference sample and areconstructed reference sample of a left adjacent line located in ahorizontal direction of a current sample and reconstructed referencesamples of an upper adjacent line in a vertical direction of the currentsample is described with reference to FIG. 23 , the disclosure is notlimited thereto, and it will be obvious to one of ordinary skill in theart that weighted prediction may be performed by using, instead of thereconstructed reference sample, an original reference sample of the leftadjacent line located in the horizontal direction of the current sampleand the original reference samples of the upper adjacent line in thevertical direction of the current sample.

FIG. 24 is a diagram for describing a process, performed by an imagedecoding apparatus, of performing position-based intra prediction on acurrent sample when an intra prediction mode of a current block is oneof a DC mode, a planar mode, and a vertical mode.

Referring to FIG. 24 , the image decoding apparatus 100 may determine afinal prediction value P′(x,y) of a current sample 2405 by using anintermediate prediction sample value P(x,y) of the current sample 2405in a current block 2400, a sample value R_(−1,y) of a left adjacentreference sample 2415 of the current sample 2405, a sample valueR_(−1,−1) of a top left adjacent reference sample 2420 of the currentblock 2400, and a sample value R_(x,−1) of an upper adjacent referencesample 2410 of the current sample 2405.

Here, the image decoding apparatus 100 may determine an intermediateprediction sample value P(x,y) of the current sample 2405 based on intraprediction according to an intra prediction mode of a current blockaccording to the related art.

For example, the image decoding apparatus 100 may determine a finalprediction sample value P′(x,y) of the current sample based on Equation13 as below.

P′(x,y)=(wL=R _(−1,y) +wT×R _(x,−1) −wTL×R_(−1,−1)+(64−wL−wT+wTL)×P(x,y)+32)>>6  [Equation 13]

Here, when an intra prediction mode of a current block is one of a DCmode, a planar mode, a horizontal mode or a vertical mode, wT, wL, andwTL may be determined according to Equation 14 as below. Here, width andheight may respectively denote a width and height of the current block.

wT=32>>((y<<1)>>shift),wL=32>>((x<<1)>>shift),

wTL=(predModeIntra==INTRA_DC)?(wL>>4)+(wT>>4):0,

shift=(log₂(width)−2+log₂(height)−2+2)>>2  [Equation 14]

predModeIntra may denote an intra prediction mode of the current block,and INTRA_DC may denote a DC mode.

FIG. 25 is a diagram for describing a process, performed by an imagedecoding apparatus, of performing position-based intra prediction on acurrent sample when an intra prediction mode of a current block is adiagonal mode in a bottom left direction.

Referring to FIG. 25 , the image decoding apparatus 100 may generate aprediction value P(x′, y′) of a current sample 2505 by using anintermediate prediction value P(x′, y′) of the current sample 2505, asample value R_((−1,y)) of a sample 2510 located on a line in a bottomleft diagonal direction from the current sample 2505, and a sample valueR_((x,−1)) of a sample 2515 located on a line opposite to the line inthe bottom left diagonal direction from the current sample 2505.

Here, the image decoding apparatus 100 may determine the intermediateprediction sample value P(x′, y′) of the current sample 2505 based onintra prediction according to an intra prediction mode of a currentblock according to the related art (a diagonal mode in the bottom leftdirection). The diagonal mode in the bottom left direction may be modeNo. 2.

For example, the image decoding apparatus 100 may determine a finalprediction sample value P′(x′,y′) of the current sample based onEquations 15 and 16 as below.

P′(x′,y′)=(wL×R _(−1,y) +wT×R _(x,−1) −wTL×R_(−1,−1)+(64−wL−wT+wTL)×P(x′,y′)+32)>>6  [Equation 15]

wT=16>>((y′<<1)>>shift)wL=16>>((x′<<1)shift). wTL=0x=x′+y′+1,y=x′+y′+1[Equation 16]

FIG. 26 is a diagram for describing a process, performed by an imagedecoding apparatus, of performing position-based intra prediction on acurrent sample when an intra prediction mode of a current block is adiagonal mode in a top right direction.

Referring to FIG. 26 , the image decoding apparatus 100 may generate aprediction value P′(x′,y′) of a current sample 2605 by using anintermediate prediction value P(x′,y′) of the current sample 2605, asample value R_((x,−1)) of a sample 2610 on a line in a top rightdiagonal direction from the current sample 2605, and a sample valueR_((−1,y)) of a sample 2615 located on a line opposite to the line inthe top right diagonal direction from the current sample 2605.

Here, the image decoding apparatus 100 may determine an intermediateprediction sample value P(x′,y′) of the current sample 2605 based onintra prediction according to an intra prediction mode of a currentblock according to the related art (a diagonal mode in the top rightdirection). The diagonal mode in the top right direction may be mode No.66.

For example, the image decoding apparatus 100 may determine a finalprediction sample value P′(x′,y′) of the current sample based onEquations 17 and 18 as below.

P′(x′,y′)=(wL×R _(−1,y) +wT×R _(x,−1) −wTL×R _(−1,−1)+(64−w _(L)−wT+wTL)×P(x′,y′)+32)>>6  [Equation 17]

wT=16>>((y′<<1)>>shift). wL=16>>((x′<<1)>>shift). wTL=0

x=x′+y′+1,y=x′+y′+1  [Equation 18]

FIG. 27 is a diagram for describing a process, performed by an imagedecoding apparatus, of performing position-based intra prediction on acurrent sample when an intra prediction mode of a current block is anangular mode adjacent to a diagonal mode in a bottom left direction.

Referring to FIG. 27 , the image decoding apparatus 100 may generate aprediction value of a current sample 2705 by using an intermediateprediction value P(x′,y′) of the current sample 2705 in a current block2700 and a sample value R_((x,−1)) of a sample 2710 located in anopposite direction to a line in a bottom left direction according to anangular mode, from the current sample 2705. An angular mode adjacent toa diagonal mode in the bottom left direction may be one of modes Nos. 3to 10.

Here, the image decoding apparatus 100 may determine an intermediateprediction sample value P(x′,y′) of the current sample 2705 based onintra prediction according to an intra prediction mode of a currentblock according to the related art (an angular mode in the bottom leftdirection).

For example, the image decoding apparatus 100 may determine a finalprediction sample value P′(x′,y′) of the current sample based onEquations 19 and 20 as below.

P′(x′,y′)=(wL×R _(−1,y) +wT×R _(x,−1) −wTL×R_(−1,−1)+(64−wL−wT+wTL)×P(x′,y′)+32)>>6  [Equation 19]

wT=32>>((y′<<1)>>shift),wL=0,wTL=0  [Equation 20]

When a coordinate x indicates a fractional sample position of, a samplevalue of Rx,⁻¹ may be determined based on sample values of two adjacentinteger samples and a distance between the coordinate x and an integersample adjacent to the coordinate x.

FIG. 28 is a diagram for describing a process, performed by an imagedecoding apparatus, of performing position-based intra prediction on acurrent sample when an intra prediction mode of a current block is anangular mode adjacent to a diagonal mode in a top right direction.

Referring to FIG. 28 , the image decoding apparatus 100 may generate aprediction value of a current sample 2805 by using an intermediateprediction value P(x′,y′) of the current sample 2805 and a sample valueR_((−1,y)) of a sample 2810 located in an opposite direction to a linein a top right direction according to an angular mode, from the currentsample 2805. An angular mode adjacent to a diagonal mode in the topright direction may be one of modes Nos. 58 to 65.

Here, the image decoding apparatus 100 may determine an intermediateprediction sample value P(x′,y′) of the current sample 2805 based onintra prediction according to an intra prediction mode of a currentblock according to the related art (an angular mode in the top rightdirection).

For example, the image decoding apparatus 100 may determine a finalprediction sample value P′(x′,y′) of the current sample based onEquations 21 and 22 as below.

P′(x′,y′)=(wL×R _(−1,y) +wT×R _(x,−1) −wTL×R_(−1,−1)+(64−wL−wT+wTL)×P(x′,y′)+32)>>6  [Equation 21]

wL=>((x′<<1)shift),wT=0,wTL=0  [Equation 22]

When a coordinate y indicates a fractional position sample, a samplevalue of R_(−1,y) may be determined based on sample values of twoadjacent integer samples and a distance between the coordinate y and aninteger sample adjacent to the coordinate y.

The method and apparatus for determining a filtering reference sampleand a weight of a filter based on the assumption that a current block isa square and for adaptively performing intra prediction based on thefiltering reference sample and the weight of the filter are describedabove with reference to FIGS. 17 through 28 . However, it will beobvious to one of ordinary skill in the art that, also when a currentblock is a rectangle, the image decoding apparatus 100 may similarlydetermine a filtering reference sample to be filtered and a weight of afilter and adaptively perform intra prediction based on the filteringreference sample and the weight of the filter. Here, when a currentblock is a rectangle having a W×H size (W is a width, and H is aheight), the number of samples of an upper adjacent reference line ofthe current block may be 2W, and the number of samples of a leftadjacent reference line may be 2H. However, the disclosure is notlimited thereto, and when a current block is a rectangle having a W×Hsize (W is a width, and H is a height), the number of reference samplesof an upper adjacent reference line of the current block may be W+H, andthe number of reference samples of a left adjacent reference line may beW+H.

FIG. 29 is a diagram for describing that an encoding (decoding) orderamong coding units is determined to be in a forward or backwarddirection based on an encoding order flag and that a right or upperreference line may be used for intra prediction according to thedetermined encoding (decoding) order, according to an embodiment of thepresent disclosure.

Referring to FIG. 29 , a coding tree unit 2950 is split into a pluralityof coding units 2956, 2958, 2960, 2962, 2968, 2970, 2972, 2974, 2980,2982, 2984, and 2986. The coding tree unit 2950 corresponds to a toplevel node 2900 of a tree structure. Also, the plurality of coding units2956, 2958, 2960, 2962, 2968, 2970, 2972, 2974, 2980, 2982, 2984, and2986 respectively correspond to a plurality of nodes 2906, 2908, 2910,2912, 2918, 2920, 2922, 2924, 2930, 2932, 2934, and 2936. Top encodingorder flags 2902, 2914, and 2926 indicating a coding order in a treestructure respectively correspond to arrows 2952, 2964, and 2976, andtop encoding order flags 2904, 2916, and 2928 respectively correspond toarrows 2954, 2966, and 2978.

The top encoding order flags indicate an encoding order of two codingunits at top ends from among coding units of an identical depth. When atop encoding order flag is 0, encoding is performed in a forwarddirection. In contrast, when a top encoding order flag is 1, encoding isperformed in a backward direction.

Likewise, bottom encoding order flags indicate an encoding order of twocoding units at bottom ends from among for coding units of an identicaldepth. When a bottom encoding order flag is 0, encoding is performed ina forward direction. In contrast, when a bottom encoding order flag is1, encoding is performed in a backward direction.

For example, as the top encoding order flag 2914 is 0, an encoding orderbetween the coding units 2968 and 2970 is determined to be from the leftto the right, which is a forward direction. Also, as the bottom encodingorder flag 2916 is 1, an encoding order between the coding units 2972and 2974 is determined from the right to the left, which is a backwarddirection.

According to an embodiment, a top encoding order flag and a bottomencoding order flag may be set to have an equal value. For example, whenthe top encoding order flag 2902 is determined to be 1, the bottomencoding order flag 2904 corresponding to the top encoding order flag2902 may also be determined to be 1. As values of the top encoding orderflag and the bottom encoding order flag are determined to be 1 bit, aninformation amount of encoding order information is reduced.

According to an embodiment, a top encoding order flag and a bottomencoding order flag of a current coding unit may be determined byreferring to at least one of a top encoding order flag and a bottomencoding order flag that are applied to a coding unit having a deeperdepth than the current coding unit. For example, the top encoding orderflag 2926 and the bottom encoding order flag 2928 applied to the codingunits 2980, 2982, 2984, and 2986 may be determined based on the bottomencoding order flag 2916 applied to the coding units 2972 and 2974.Accordingly, the top encoding order flag 2926 and the bottom encodingorder flag 2928 may be determined to have an equal value to that of theencoding order flag 2916. As values of the top encoding order flag andthe bottom encoding order flag are determined from an upper coding unitof the current coding unit, encoding order information is not obtainedfrom a bitstream. Accordingly, an information amount of encoding orderinformation is reduced.

Here, as data of samples included in the right adjacent coding unit 2958decoded earlier than the current coding unit 2986 and data of samplesincluded in the upper adjacent coding units 2980 and 2982 are available,the image decoding apparatus 100 may perform prediction according to anembodiment of the present disclosure by using the data of the samplesincluded in the right adjacent coding unit 2958 (right reference line)and the data of the samples included in the upper adjacent coding units2980 and 2982 (upper reference line).

While a method and apparatus for determining a filtering referencesample to be filtered and a weight of a filter and for adaptivelyperforming intra prediction based on the filtering reference sample andthe weight of the filter, and performing of intra prediction based onoriginal reference samples adjacent to an upper or left corner of acurrent block by assuming performing encoding and decoding according toan encoding or decoding order of a coding unit according to the relatedart, are described with reference to FIGS. 17 through 28 , thedisclosure is not limited thereto, and it will be obvious to one ofordinary skill in the art that intra prediction may be performed basedon original reference samples adjacent to an upper or right corner ofthe current block when an encoding/decoding order among some adjacentcoding units is in a backward order as in FIG. 29 .

According to the various embodiments of the present disclosure, asinformation of reference samples which are not used in intra predictionaccording to the related art is used, prediction accuracy may beimproved, and also, the effect of reference smoothing is applied toreconstructed reference samples in a process of reconstructing anoriginal reference sample, and thus, prediction accuracy may beimproved. In addition, as various prediction blocks may be generated byselectively using a reconstructed reference sample and an originalreference sample, a more efficient prediction block may be selected,thereby improving prediction accuracy. Also, according to variousembodiments of the present disclosure, a prediction block having anatural pattern is generated, and thus, an abrupt prediction error maybe corrected, thereby improving the transform efficiency.

This disclosure has been particularly shown and described with referenceto embodiments thereof. It will be understood by those of ordinary skillin the art that various changes in form and details may be made thereinwithout departing from the spirit and scope of the present disclosure asdefined by the appended claims. Thus, the disclosed embodiments shouldbe considered in a descriptive sense only and not for purposes oflimitation. The scope of the present disclosure is defined not by thedetailed description of the disclosure but by the appended claims, andall differences within the scope will be construed as being included inthe present disclosure.

The embodiments of the present disclosure described above may be writtenas computer programs and can be implemented in general-use digitalcomputers that execute the programs using a computer-readable recordingmedium. Examples of the computer-readable recording medium includestorage media such as magnetic storage media (e.g., read-only memories,floppy disks, or hard disks), optical recording media (e.g., CD-ROMs orDVDs), etc.

1. An image decoding method, comprising: obtaining an intra predictionsample of a current sample in a current block using a first referencesample corresponding to the current sample, wherein the first referencesample is determined based on a position of the current sample and anintra prediction mode of the current block; when the intra predictionmode of the current block is a first angular mode from among upperprediction directions including a upper-right prediction direction,determining a second reference sample from among a plurality ofreference samples in a left side of the current block based on theposition of the current sample and the first angular mode, determining asecond weight for the second reference sample based on a X-coordinate ofthe position of the current sample, and obtaining a modified intraprediction sample of the current sample based on the second referencesample, the intra prediction sample of the current sample, the secondweight for the second reference sample and a first weight for the intraprediction sample of the current sample; when the intra prediction modeof the current block is a second angular mode from among left predictiondirections including a lower-left prediction direction, determining athird reference sample from among the plurality of reference samples inan upper side of the current block based on the position of the currentsample and the second angular mode, determining a third weight for thethird reference sample based on a Y-coordinate of the position of thecurrent sample, and obtaining a modified intra prediction sample of thecurrent sample based on the third reference sample, the intra predictionsample of the current sample, the third weight for the third referencesample and the first weight for the intra prediction sample of thecurrent sample; and reconstructing the current block using the modifiedintra prediction sample of the current sample and a residual sample ofthe current sample, wherein the current block is among a plurality ofblocks split from an upper block.
 2. An image encoding method,comprising: obtaining an intra prediction sample of a current sample ina current block using a first reference sample corresponding to thecurrent sample, wherein the first reference sample is determined basedon a position of the current sample and an intra prediction mode of thecurrent block; when the intra prediction mode of the current block is afirst angular mode from among upper prediction directions including aupper-right prediction direction, determining a second reference samplefrom among a plurality of reference samples in a left side of thecurrent block based on the position of the current sample and the firstangular mode, determining a second weight for the second referencesample based on a X-coordinate of the position of the current sample,and obtaining a modified intra prediction sample of the current samplebased on the second reference sample, the intra prediction sample of thecurrent sample, the second weight for the second reference sample and afirst weight for the intra prediction sample of the current sample; whenthe intra prediction mode of the current block is a second angular modefrom among left prediction directions including a lower-left predictiondirection, determining a third reference sample from among the pluralityof reference samples in an upper side of the current block based on theposition of the current sample and the second angular mode, determininga third weight for the third reference sample based on a Y-coordinate ofthe position of the current sample, and obtaining a modified intraprediction sample of the current sample based on the third referencesample, the intra prediction sample of the current sample, the thirdweight for the third reference sample and the first weight for the intraprediction sample of the current sample; and encoding the current blockusing the modified intra prediction sample of the current sample,wherein the current block is among a plurality of blocks split from anupper block.
 3. A non-transitory computer-readable medium for recordinga bitstream, the bitstream comprising: an encoded data of a currentblock, wherein the encoded data of the current block is obtained by:obtaining an intra prediction sample of a current sample in the currentblock using a first reference sample corresponding to the currentsample, wherein the first reference sample is determined based on aposition of the current sample and an intra prediction mode of thecurrent block; when the intra prediction mode of the current block is afirst angular mode from among upper prediction directions including aupper-right prediction direction, determining a second reference samplefrom among a plurality of reference samples in a left side of thecurrent block based on the position of the current sample and the firstangular mode, determining a second weight for the second referencesample based on a X-coordinate of the position of the current sample,and obtaining a modified intra prediction sample of the current samplebased on the second reference sample, the intra prediction sample of thecurrent sample, the second weight for the second reference sample and afirst weight for the intra prediction sample of the current sample; whenthe intra prediction mode of the current block is a second angular modefrom among left prediction directions including a lower-left predictiondirection, determining a third reference sample from among the pluralityof reference samples in an upper side of the current block based on theposition of the current sample and the second angular mode, determininga third weight for the third reference sample based on a Y-coordinate ofthe position of the current sample, and obtaining a modified intraprediction sample of the current sample based on the third referencesample, the intra prediction sample of the current sample, the thirdweight for the third reference sample and the first weight for the intraprediction sample of the current sample; and encoding the current blockusing the modified intra prediction sample of the current sample,wherein the current block is among a plurality of blocks split from anupper block.